las reglas PREROUTING establecidas por el calico están en cualquier lugar

Question

Estoy confundido. Esta es mi configuración de tabla nat de iptables

[root@k8s-51 woniu.zhang]# iptables -t nat -L -v  --line-numbers

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    4566K  396M cali-PREROUTING  all  --  any    any     anywhere             anywhere             /* cali:6gwbT8clXdHdC1b1 */
2    4567K  396M KUBE-SERVICES  all  --  any    any     anywhere             anywhere             /* kubernetes service portals */
3     7687  465K CNI-HOSTPORT-DNAT  all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
4     3923  236K DOCKER     all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
5     142K   12M            all  --  any    any     anywhere             anywhere
6     142K   12M            all  --  any    any     anywhere             anywhere

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    7901K  549M cali-OUTPUT  all  --  any    any     anywhere             anywhere             /* cali:tVnHkvAo15HuiPy0 */
2    7902K  549M KUBE-SERVICES  all  --  any    any     anywhere             anywhere             /* kubernetes service portals */
3     555K   33M CNI-HOSTPORT-DNAT  all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
4       67  4237 DOCKER     all  --  any    any     anywhere            !loopback/8           ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    6657K  469M cali-POSTROUTING  all  --  any    any     anywhere             anywhere             /* cali:O3lYWMrLQYEMJtB5 */
2        0     0 MASQUERADE  all  --  any    !docker0  172.17.0.0/16        anywhere
3    7256K  507M CNI-HOSTPORT-MASQ  all  --  any    any     anywhere             anywhere             /* CNI portfwd requiring masquerade */
4    8073K  560M KUBE-POSTROUTING  all  --  any    any     anywhere             anywhere             /* kubernetes postrouting rules */

Chain CNI-HOSTPORT-DNAT (2 references)
num   pkts bytes target     prot opt in     out     source               destination

Chain CNI-HOSTPORT-MASQ (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1       11   660 MASQUERADE  all  --  any    any     anywhere             anywhere             mark match 0x2000/0x2000

Chain CNI-HOSTPORT-SETMARK (0 references)
num   pkts bytes target     prot opt in     out     source               destination
1       11   660 MARK       all  --  any    any     anywhere             anywhere             /* CNI portfwd masquerade mark */ MARK or 0x2000

Chain DOCKER (2 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 RETURN     all  --  docker0 any     anywhere             anywhere

Chain KUBE-FIREWALL (0 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 KUBE-MARK-DROP  all  --  any    any     anywhere             anywhere

Chain KUBE-KUBELET-CANARY (0 references)
num   pkts bytes target     prot opt in     out     source               destination

Chain KUBE-LOAD-BALANCER (0 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 KUBE-MARK-MASQ  all  --  any    any     anywhere             anywhere

Chain KUBE-MARK-DROP (1 references)
num   pkts bytes target     prot opt in     out     source               destination

Chain KUBE-MARK-MASQ (3 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 MARK       all  --  any    any     anywhere             anywhere             MARK or 0x4000

Chain KUBE-NODE-PORT (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 KUBE-MARK-MASQ  tcp  --  any    any     anywhere             anywhere             /* Kubernetes nodeport TCP port for masquerade purpose */ match-set KUBE-NODE-PORT-TCP dst

Chain KUBE-POSTROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 MASQUERADE  all  --  any    any     anywhere             anywhere             /* Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose */ match-set KUBE-LOOP-BACK dst,dst,src
2        0     0 RETURN     all  --  any    any     anywhere             anywhere             mark match ! 0x4000/0x4000
3        0     0 MARK       all  --  any    any     anywhere             anywhere             MARK xor 0x4000
4        0     0 MASQUERADE  all  --  any    any     anywhere             anywhere             /* kubernetes service traffic requiring SNAT */

Chain KUBE-SERVICES (2 references)
num   pkts bytes target     prot opt in     out     source               destination
1        0     0 KUBE-MARK-MASQ  all  --  any    any     anywhere             anywhere             /* Kubernetes service cluster ip + port for masquerade purpose */ match-set KUBE-CLUSTER-IP dst,dst
2        0     0 KUBE-NODE-PORT  all  --  any    any     anywhere             anywhere             ADDRTYPE match dst-type LOCAL
3        0     0 ACCEPT     all  --  any    any     anywhere             anywhere             match-set KUBE-CLUSTER-IP dst,dst

Chain cali-OUTPUT (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1    7901K  549M cali-fip-dnat  all  --  any    any     anywhere             anywhere             /* cali:GBTAv2p5CwevEyJm */

Chain cali-POSTROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1    7933K  551M cali-fip-snat  all  --  any    any     anywhere             anywhere             /* cali:Z-c7XtVd2Bq7s_hA */
2    7933K  551M cali-nat-outgoing  all  --  any    any     anywhere             anywhere             /* cali:nYKhEzDlr11Jccal */
3        0     0 MASQUERADE  all  --  any    tunl0   anywhere             anywhere             /* cali:JHlpT-eSqR1TvyYm */ ADDRTYPE match src-type !LOCAL limit-out ADDRTYPE match src-type LOCAL

Chain cali-PREROUTING (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1    4566K  396M cali-fip-dnat  all  --  any    any     anywhere             anywhere             /* cali:r6XmIziWUJsdOK6Z */

Chain cali-fip-dnat (2 references)
num   pkts bytes target     prot opt in     out     source               destination

Chain cali-fip-snat (1 references)
num   pkts bytes target     prot opt in     out     source               destination

Chain cali-nat-outgoing (1 references)
num   pkts bytes target     prot opt in     out     source               destination
1     2185  131K MASQUERADE  all  --  any    any     anywhere             anywhere             /* cali:Dw4T8UWPnCLxRJiI */ match-set cali40masq-ipam-pools src ! match-set cali40all-ipam-pools dst

Los resultados de iptables-save se muestran a continuación

[root@k8s-51 woniu.zhang]# iptables-save

# Completed on Tue Jan 12 11:11:06 2021
# Generated by iptables-save v1.4.21 on Tue Jan 12 11:11:06 2021
*nat
:PREROUTING ACCEPT [4:463]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [25:1810]
:POSTROUTING ACCEPT [25:1810]
:CNI-HOSTPORT-DNAT - [0:0]
:CNI-HOSTPORT-MASQ - [0:0]
:CNI-HOSTPORT-SETMARK - [0:0]
:DOCKER - [0:0]
:KUBE-FIREWALL - [0:0]
:KUBE-KUBELET-CANARY - [0:0]
:KUBE-LOAD-BALANCER - [0:0]
:KUBE-MARK-DROP - [0:0]
:KUBE-MARK-MASQ - [0:0]
:KUBE-NODE-PORT - [0:0]
:KUBE-POSTROUTING - [0:0]
:KUBE-SERVICES - [0:0]
:cali-OUTPUT - [0:0]
:cali-POSTROUTING - [0:0]
:cali-PREROUTING - [0:0]
:cali-fip-dnat - [0:0]
:cali-fip-snat - [0:0]
:cali-nat-outgoing - [0:0]
-A PREROUTING -m comment --comment "cali:6gwbT8clXdHdC1b1" -j cali-PREROUTING
-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A PREROUTING -m addrtype --dst-type LOCAL -j CNI-HOSTPORT-DNAT
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A PREROUTING
-A PREROUTING
-A OUTPUT -m comment --comment "cali:tVnHkvAo15HuiPy0" -j cali-OUTPUT
-A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
-A OUTPUT -m addrtype --dst-type LOCAL -j CNI-HOSTPORT-DNAT
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -m comment --comment "cali:O3lYWMrLQYEMJtB5" -j cali-POSTROUTING
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -m comment --comment "CNI portfwd requiring masquerade" -j CNI-HOSTPORT-MASQ
-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
-A CNI-HOSTPORT-MASQ -m mark --mark 0x2000/0x2000 -j MASQUERADE
-A CNI-HOSTPORT-SETMARK -m comment --comment "CNI portfwd masquerade mark" -j MARK --set-xmark 0x2000/0x2000
-A DOCKER -i docker0 -j RETURN
-A KUBE-FIREWALL -j KUBE-MARK-DROP
-A KUBE-LOAD-BALANCER -j KUBE-MARK-MASQ
-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
-A KUBE-NODE-PORT -p tcp -m comment --comment "Kubernetes nodeport TCP port for masquerade purpose" -m set --match-set KUBE-NODE-PORT-TCP dst -j KUBE-MARK-MASQ
-A KUBE-POSTROUTING -m comment --comment "Kubernetes endpoints dst ip:port, source ip for solving hairpin purpose" -m set --match-set KUBE-LOOP-BACK dst,dst,src -j MASQUERADE
-A KUBE-POSTROUTING -m mark ! --mark 0x4000/0x4000 -j RETURN
-A KUBE-POSTROUTING -j MARK --set-xmark 0x4000/0x0
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -j MASQUERADE
-A KUBE-SERVICES -m comment --comment "Kubernetes service cluster ip + port for masquerade purpose" -m set --match-set KUBE-CLUSTER-IP dst,dst -j KUBE-MARK-MASQ
-A KUBE-SERVICES -m addrtype --dst-type LOCAL -j KUBE-NODE-PORT
-A KUBE-SERVICES -m set --match-set KUBE-CLUSTER-IP dst,dst -j ACCEPT
-A cali-OUTPUT -m comment --comment "cali:GBTAv2p5CwevEyJm" -j cali-fip-dnat
-A cali-POSTROUTING -m comment --comment "cali:Z-c7XtVd2Bq7s_hA" -j cali-fip-snat
-A cali-POSTROUTING -m comment --comment "cali:nYKhEzDlr11Jccal" -j cali-nat-outgoing
-A cali-POSTROUTING -o tunl0 -m comment --comment "cali:JHlpT-eSqR1TvyYm" -m addrtype ! --src-type LOCAL --limit-iface-out -m addrtype --src-type LOCAL -j MASQUERADE
-A cali-PREROUTING -m comment --comment "cali:r6XmIziWUJsdOK6Z" -j cali-fip-dnat
-A cali-nat-outgoing -m comment --comment "cali:Dw4T8UWPnCLxRJiI" -m set --match-set cali40masq-ipam-pools src -m set ! --match-set cali40all-ipam-pools dst -j MASQUERADE
COMMIT

Estoy confundido con las dos reglas anteriores en cualquier lugar:

[root@k8s-51 woniu.zhang]# iptables -t nat -L -v  --line-numbers

Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1    4566K  396M cali-PREROUTING  all  --  any    any     anywhere             anywhere             /* cali:6gwbT8clXdHdC1b1 */
2    4567K  396M KUBE-SERVICES  all  --  any    any     anywhere             anywhere             /* kubernetes service portals */

La primera regla es aceptar todo el tráfico, ¿cómo y cuándo coinciden las siguientes reglas?

Answer 1

No, la primera regla no acepta todo el tráfico. Simplemente dirige el paquete a otra cadena. Más aún, si no coincide ninguna regla, o se acepta el paquete, ese paquete terminado solo viaja a través de esta tabla y esta cadena maestra, pero aún debe pasar por otras cadenas de la tabla y otras tablas.

Para este caso: parece que solo natse utiliza la tabla, donde los paquetes entrantes viajan reglas en el siguiente orden:

1. entra en PREROUTINGcadena,
2. después de la regla -A PREROUTING -m comment --comment "cali:6gwbT8clXdHdC1b1" -j cali-PREROUTINGsalta acali-PREROUTING
3. hay una regla -A cali-PREROUTING -m comment --comment "cali:r6XmIziWUJsdOK6Z" -j cali-fip-dnata la que saltacali-fip-dnat
4. no hay reglas en esa cadena, por lo que eventualmente regresa a la cadena PREROUTINGy procesa la siguiente regla
5. la regla -A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICESlo pone enKUBE-SERVICES

Allí comienza un procesamiento útil. El paquete se está marcando o se acepta (y si se trata de un firewall completo, no se realiza ningún procesamiento adicional).

etcétera.

Tenga en cuenta también que este recorrido se realiza sólo para el primer paquete de una "conexión" (flujo bidireccional de paquetes relacionados). Cuando Linux determina el destino de este paquete, se convierte en el destino de esta "conexión". Instala un registro dinámico en una tabla de conntrack especial y, si algún paquete siguiente coincide con esta conexión mediante conntrack, se procesa de acuerdo con el registro dinámico en conntrack y no se procesa completamente a través de las reglas del firewall. El registro dinámico en conntrack se elimina después del final de la conexión, ya sea al cerrar (como TCP FIN o RST) o después del tiempo de espera.