☰

カリコが設定したPREROUTINGルールはどこにでもある

tag-icon • tag-icon iptables calico
カリコが設定したPREROUTINGルールはどこにでもある

混乱しています。これは私のiptables natテーブル設定です

[root@k8s-51 woniu.zhang]# iptables -t nat -L -v  --line-numbers

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    4566K  396M cali-PREROUTING  all  --  any    any     anywhere             anywhere             /* cali:6gwbT8clXdHdC1b1 */
2    4567K  396M KUBE-SERVICES  all  --  any    any     anywhere             anywhere             /* kubernetes service portals */
3     7687  465K CNI-HOSTPORT-DNAT  all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
4     3923  236K DOCKER     all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
5     142K   12M            all  --  any    any     anywhere             anywhere
6     142K   12M            all  --  any    any     anywhere             anywhere

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    7901K  549M cali-OUTPUT  all  --  any    any     anywhere             anywhere             /* cali:tVnHkvAo15HuiPy0 */
2    7902K  549M KUBE-SERVICES  all  --  any    any     anywhere             anywhere             /* kubernetes service portals */
3     555K   33M CNI-HOSTPORT-DNAT  all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
4       67  4237 DOCKER     all  --  any    any     anywhere            !loopback/8           ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    6657K  469M cali-POSTROUTING  all  --  any    any     anywhere             anywhere             /* cali:O3lYWMrLQYEMJtB5 */
2        0     0 MASQUERADE  all  --  any    !docker0  172.17.0.0/16        anywhere
3    7256K  507M CNI-HOSTPORT-MASQ  all  --  any    any     anywhere             anywhere             /* CNI portfwd requiring masquerade */
4    8073K  560M KUBE-POSTROUTING  all  --  any    any     anywhere             anywhere             /* kubernetes postrouting rules */

Chain CNI-HOSTPORT-DNAT (2 references)
num   pkts bytes target     prot opt in     out     source               destination

Chain CNI-HOSTPORT-MASQ (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1       11   660 MASQUERADE  all  --  any    any     anywhere             anywhere             mark match 0x2000/0x2000

Chain CNI-HOSTPORT-SETMARK (0 references)
num   pkts bytes target     prot opt in     out     source               destination
1       11   660 MARK       all  --  any    any     anywhere             anywhere             /* CNI portfwd masquerade mark */ MARK or 0x2000

Chain DOCKER (2 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 RETURN     all  --  docker0 any     anywhere             anywhere

Chain KUBE-FIREWALL (0 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 KUBE-MARK-DROP  all  --  any    any     anywhere             anywhere

Chain KUBE-KUBELET-CANARY (0 references)
num   pkts bytes target     prot opt in     out     source               destination

Chain KUBE-LOAD-BALANCER (0 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 KUBE-MARK-MASQ  all  --  any    any     anywhere             anywhere

Chain KUBE-MARK-DROP (1 references)
num   pkts bytes target     prot opt in     out     source               destination

Chain KUBE-MARK-MASQ (3 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 MARK       all  --  any    any     anywhere             anywhere             MARK or 0x4000

Chain KUBE-NODE-PORT (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 KUBE-MARK-MASQ  tcp  --  any    any     anywhere             anywhere             /* Kubernetes nodeport TCP port for masquerade purpose */ match-set KUBE-NODE-PORT-TCP dst

Chain KUBE-POSTROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 MASQUERADE  all  --  any    any     anywhere             anywhere             /* Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose */ match-set KUBE-LOOP-BACK dst,dst,src
2        0     0 RETURN     all  --  any    any     anywhere             anywhere             mark match ! 0x4000/0x4000
3        0     0 MARK       all  --  any    any     anywhere             anywhere             MARK xor 0x4000
4        0     0 MASQUERADE  all  --  any    any     anywhere             anywhere             /* kubernetes service traffic requiring SNAT */

Chain KUBE-SERVICES (2 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 KUBE-MARK-MASQ  all  --  any    any     anywhere             anywhere             /* Kubernetes service cluster ip + port for masquerade purpose */ match-set KUBE-CLUSTER-IP dst,dst
2        0     0 KUBE-NODE-PORT  all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
3        0     0 ACCEPT     all  --  any    any     anywhere             anywhere             match-set KUBE-CLUSTER-IP dst,dst

Chain cali-OUTPUT (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1    7901K  549M cali-fip-dnat  all  --  any    any     anywhere             anywhere             /* cali:GBTAv2p5CwevEyJm */

Chain cali-POSTROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1    7933K  551M cali-fip-snat  all  --  any    any     anywhere             anywhere             /* cali:Z-c7XtVd2Bq7s_hA */
2    7933K  551M cali-nat-outgoing  all  --  any    any     anywhere             anywhere             /* cali:nYKhEzDlr11Jccal */
3        0     0 MASQUERADE  all  --  any    tunl0   anywhere             anywhere             /* cali:JHlpT-eSqR1TvyYm */ ADDRTYPE match src-type !LOCAL limit-out ADDRTYPE match src-type LOCAL

Chain cali-PREROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1    4566K  396M cali-fip-dnat  all  --  any    any     anywhere             anywhere             /* cali:r6XmIziWUJsdOK6Z */

Chain cali-fip-dnat (2 references)
num   pkts bytes target     prot opt in     out     source               destination

Chain cali-fip-snat (1 references)
num   pkts bytes target     prot opt in     out     source               destination

Chain cali-nat-outgoing (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1     2185  131K MASQUERADE  all  --  any    any     anywhere             anywhere             /* cali:Dw4T8UWPnCLxRJiI */ match-set cali40masq-ipam-pools src ! match-set cali40all-ipam-pools dst

iptables-saveの結果は以下のとおりです

[root@k8s-51 woniu.zhang]# iptables-save

# Completed on Tue Jan 12 11:11:06 2021
# Generated by iptables-save v1.4.21 on Tue Jan 12 11:11:06 2021
*nat
:PREROUTING ACCEPT [4:463]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [25:1810]
:POSTROUTING ACCEPT [25:1810]
:CNI-HOSTPORT-DNAT - [0:0]
:CNI-HOSTPORT-MASQ - [0:0]
:CNI-HOSTPORT-SETMARK - [0:0]
:DOCKER - [0:0]
:KUBE-FIREWALL - [0:0]
:KUBE-KUBELET-CANARY - [0:0]
:KUBE-LOAD-BALANCER - [0:0]
:KUBE-MARK-DROP - [0:0]
:KUBE-MARK-MASQ - [0:0]
:KUBE-NODE-PORT - [0:0]
:KUBE-POSTROUTING - [0:0]
:KUBE-SERVICES - [0:0]
:cali-OUTPUT - [0:0]
:cali-POSTROUTING - [0:0]
:cali-PREROUTING - [0:0]
:cali-fip-dnat - [0:0]
:cali-fip-snat - [0:0]
:cali-nat-outgoing - [0:0]
-A PREROUTING -m comment --comment "cali:6gwbT8clXdHdC1b1" -j cali-PREROUTING
-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A PREROUTING -m addrtype --dst-type LOCAL -j CNI-HOSTPORT-DNAT
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A PREROUTING
-A PREROUTING
-A OUTPUT -m comment --comment "cali:tVnHkvAo15HuiPy0" -j cali-OUTPUT
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT -m addrtype --dst-type LOCAL -j CNI-HOSTPORT-DNAT
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -m comment --comment "cali:O3lYWMrLQYEMJtB5" -j cali-POSTROUTING
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -m comment --comment "CNI portfwd requiring masquerade" -j CNI-HOSTPORT-MASQ
-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
-A CNI-HOSTPORT-MASQ -m mark --mark 0x2000/0x2000 -j MASQUERADE
-A CNI-HOSTPORT-SETMARK -m comment --comment "CNI portfwd masquerade mark" -j MARK --set-xmark 0x2000/0x2000
-A DOCKER -i docker0 -j RETURN
-A KUBE-FIREWALL -j KUBE-MARK-DROP
-A KUBE-LOAD-BALANCER -j KUBE-MARK-MASQ
-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
-A KUBE-NODE-PORT -p tcp -m comment --comment "Kubernetes nodeport TCP port for masquerade purpose" -m set --match-set KUBE-NODE-PORT-TCP dst -j KUBE-MARK-MASQ
-A KUBE-POSTROUTING -m comment --comment "Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose" -m set --match-set KUBE-LOOP-BACK dst,dst,src -j MASQUERADE
-A KUBE-POSTROUTING -m mark ! --mark 0x4000/0x4000 -j RETURN
-A KUBE-POSTROUTING -j MARK --set-xmark 0x4000/0x0
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -j MASQUERADE
-A KUBE-SERVICES -m comment --comment "Kubernetes service cluster ip + port for masquerade purpose" -m set --match-set KUBE-CLUSTER-IP dst,dst -j KUBE-MARK-MASQ
-A KUBE-SERVICES -m addrtype --dst-type LOCAL -j KUBE-NODE-PORT
-A KUBE-SERVICES -m set --match-set KUBE-CLUSTER-IP dst,dst -j ACCEPT
-A cali-OUTPUT -m comment --comment "cali:GBTAv2p5CwevEyJm" -j cali-fip-dnat
-A cali-POSTROUTING -m comment --comment "cali:Z-c7XtVd2Bq7s_hA" -j cali-fip-snat
-A cali-POSTROUTING -m comment --comment "cali:nYKhEzDlr11Jccal" -j cali-nat-outgoing
-A cali-POSTROUTING -o tunl0 -m comment --comment "cali:JHlpT-eSqR1TvyYm" -m addrtype ! --src-type LOCAL --limit-iface-out -m addrtype --src-type LOCAL -j MASQUERADE
-A cali-PREROUTING -m comment --comment "cali:r6XmIziWUJsdOK6Z" -j cali-fip-dnat
-A cali-nat-outgoing -m comment --comment "cali:Dw4T8UWPnCLxRJiI" -m set --match-set cali40masq-ipam-pools src -m set ! --match-set cali40all-ipam-pools dst -j MASQUERADE
COMMIT

前の 2 つのどこでもルールがわかりません:

[root@k8s-51 woniu.zhang]# iptables -t nat -L -v  --line-numbers

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    4566K  396M cali-PREROUTING  all  --  any    any     anywhere             anywhere             /* cali:6gwbT8clXdHdC1b1 */
2    4567K  396M KUBE-SERVICES  all  --  any    any     anywhere             anywhere             /* kubernetes service portals */

最初のルールではすべてのトラフィックを受け入れるように設定されていますが、次のルールはいつどのように一致するのでしょうか?

答え1

いいえ、最初のルールはすべてのトラフィックを受け入れるわけではありません。パケットを別のチェーンに誘導するだけです。さらに、ルールが一致しない場合、またはパックされたパケットが受け入れられた場合、完成したパケットのみがこのテーブルとこのマスター チェーンを通過しますが、テーブルの他のチェーンと他のテーブルを通過する必要があります。

この場合、nat着信パケットが次の順序でルールを移動するテーブルのみが使用されているようです。

  1. PREROUTINGチェーンに入り、
  2. ルール-A PREROUTING -m comment --comment "cali:6gwbT8clXdHdC1b1" -j cali-PREROUTINGが破られた後cali-PREROUTING
  3. -A cali-PREROUTING -m comment --comment "cali:r6XmIziWUJsdOK6Z" -j cali-fip-dnatそこにはジャンプするルールがあるcali-fip-dnat
  4. そのチェーンにはルールがないので、最終的にはチェーンに戻りPREROUTING、次のルールを処理します。
  5. ルールは-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICESそれをKUBE-SERVICES

ここで、いくつかの有用な処理が始まります。パケットはマークされているか、受け入れられます (完全なファイアウォールの場合は、それ以上の処理は行われません)。

等々。

また、このトラバースは「接続」の最初のパケット（関連パケットの双方向ストリーム）に対してのみ実行されることに注意してください。Linux がこのパケットの運命を決定すると、それがこの「接続」の運命になります。Linux は特別な conntrack テーブルに動的レコードをインストールし、後続のパケットが conntrack によってこの接続に一致する場合、そのパケットは conntrack の動的レコードに従って処理され、ファイアウォール ルールによって完全には処理されません。conntrack の動的レコードは、クローズ（TCP FIN または RST など）またはタイムアウトによって接続が終了した後に削除されます。

関連情報