使用 iptables 將流量路由到 Kubernetes 集群

使用 iptables 將流量路由到 Kubernetes 集群

我有一個 kubernetes 叢集(目前只有 1 個節點)在我的根伺服器 (netcup) 上運行,並嘗試將流量路由到它。我在網路方面很糟糕,所以希望有人可以幫助我。

在集群上金屬磅已安裝並將 calico 作為 CNI。 Metallb 負責為 K8s-LoadBalancers 分配和公佈 IP 位址,內部運作良好。所以我可以從伺服器內部獲得負載平衡的服務。我隻公開具有固定內部 IP 的單一服務(反向代理)

現在我想透過使用以下命令將流量直接從 Metallb 路由到 IP 位址,從而將其公開給外部iptables

iptables 中應用了以下規則:

iptables -A PREROUTING -t nat -p tcp -m tcp -j DNAT --to-destination <metallb-ip> -i eth0 --destination-port 80 -m comment --comment Redirect web traffic to cluster (80)
iptables -A PREROUTING -t nat -p tcp -m tcp -j DNAT --to-destination <metallb-ip> -i eth0 --destination-port 443 -m comment --comment Redirect web traffic to cluster (443)
iptables -A POSTROUTING -t nat -p tcp -m tcp -j SNAT --to-source <public-server-ip> -o eth0 --destination-port 80 -m comment --comment Redirect web traffic from cluster (80)
iptables -A POSTROUTING -t nat -p tcp -m tcp -j SNAT --to-source <public-server-ip> -o eth0 --destination-port 443 -m comment --comment Redirect web traffic from cluster (443)

(也許這些命令不是100%正確,因為我從我的ansible腳本中提取了它們。你可以在底部找到ansible部分)

據我了解,這應該使用 DNAT 將所有傳入 TCP 流量傳送到內部 IP,並且所有傳出流量將使用 SNAT 具有公用伺服器位址。那麼對於連接客戶端來說,他應該看起來像是直接與公共 IP/連接埠對話?

不幸的是,嘗試從 Internet 存取該服務會導致以下結果:

$> curl <public-server-ip>
curl: (7) Failed to connect to <public-server-ip> port 80 after 647 ms: Network is down

從伺服器存取有效:

$> curl <metallb-ip>
404 page not found # which the expected answer since its a blank Traefik reverse proxy

nmap 連接埠掃描目前會拋出以下錯誤:

$> nmap -Pn -p 80,443 <public-server-ip>
Starting Nmap 7.93 ( https://nmap.org ) at 2022-12-15 15:57 CET
Strange SO_ERROR from connection to <public-server-ip> (50 - 'Network is down') -- bailing scan
QUITTING!

但自從我透過 SSH 連接以來它就已啟動並運行。

目前沒有ufw啟用。

我感謝任何幫助,因為正如我所說,我不太擅長網路。 ;)

安塞布爾腳本:

- name: DNAT port 80 to cluster
  iptables:
    table: nat
    chain: PREROUTING
    in_interface: eth0
    protocol: tcp
    match: tcp
    jump: DNAT # REDIRECT
    to_destination: <metallb-ip>
    destination_port: 80
    comment: Redirect web traffic to cluster (80)
  become: yes

- name: DNAT port 443 to cluster
  iptables:
    table: nat
    chain: PREROUTING
    in_interface: eth0
    protocol: tcp
    match: tcp
    destination_port: 443
    jump: DNAT # REDIRECT
    to_destination: <metallb-ip>
    comment: Redirect web traffic to cluster (443)
  become: yes 

- name: SNAT port 80 from cluster
  iptables:
    table: nat
    chain: POSTROUTING
    out_interface: eth0
    protocol: tcp
    match: tcp
    destination_port: 80
    jump: SNAT # REDIRECT
    to_source: <public-server-ip>
    comment: Redirect web traffic from cluster (443)
  become: yes

- name: SNAT port 443 from cluster
  iptables:
    table: nat
    chain: POSTROUTING
    out_interface: eth0
    protocol: tcp
    match: tcp
    destination_port: 443
    jump: SNAT # REDIRECT
    to_source: <public-server-ip>
    comment: Redirect web traffic from cluster (443)
  become: yes

- name: Configure IP Masquerading
  iptables: 
    table: nat
    chain: POSTROUTING
    out_interface: eth0
    jump: MASQUERADE

更新: 在...的幫助下文章我試著理解流程。新增了日誌記錄語句來調查哪裡出了問題。這就是我目前發現的:

為了檢查資料包遺失的位置,我新增了以下規則:

iptables -t raw -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[raw:PREROUTING]: "
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[mangle:PREROUTING]: "
iptables -t nat -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[nat:PREROUTING]: "
iptables -t mangle -A POSTROUTING -p tcp --sport 80 -j LOG --log-prefix "[mangle:PREROUTING]: "
iptables -t nat -A POSTROUTING -p tcp --sport 80 -j LOG --log-prefix "[nat:PREROUTING]: "

/var/log/kern.log在剛剛出現的rawmanglelog 語句下。所以看起來nat-step 沒有被觸發。檢查 mangle 規則: sudo iptables -t mangle -L但無法辨識這裡的任何問題。

IP 轉送已啟用:

$> sudo sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

那為什麼資料包沒有到達nat步驟呢?

$> sudo iptables -t mangle -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
cali-PREROUTING  all  --  anywhere             anywhere             /* cali:6gwbT8clXdHdC1b1 */
LOG        tcp  --  anywhere             anywhere             tcp dpt:http LOG level warning prefix "[mangle:PREROUTING]: "

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
cali-POSTROUTING  all  --  anywhere             anywhere             /* cali:O3lYWMrLQYEMJtB5 */

Chain KUBE-IPTABLES-HINT (0 references)
target     prot opt source               destination

Chain KUBE-KUBELET-CANARY (0 references)
target     prot opt source               destination

Chain KUBE-PROXY-CANARY (0 references)
target     prot opt source               destination

Chain cali-from-host-endpoint (1 references)
target     prot opt source               destination

Chain cali-to-host-endpoint (1 references)
target     prot opt source               destination

Chain cali-PREROUTING (1 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere             /* cali:6BJqBjBC7crtA-7- */ ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere             /* cali:KX7AGNd6rMcDUai6 */ mark match 0x10000/0x10000
cali-from-host-endpoint  all  --  anywhere             anywhere             /* cali:wNH7KsA3ILKJBsY9 */
ACCEPT     all  --  anywhere             anywhere             /* cali:Cg96MgVuoPm7UMRo */ /* Host endpoint policy accepted packet. */ mark match 0x10000/0x10000

Chain cali-POSTROUTING (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere             /* cali:NX-7roTexQ3fGRfU */ mark match 0x10000/0x10000
MARK       all  --  anywhere             anywhere             /* cali:nnqPh8lh2VOogSzX */ MARK and 0xfff0ffff
cali-to-host-endpoint  all  --  anywhere             anywhere             /* cali:nquN8Jw8Tz72pcBW */ ctstate DNAT
RETURN     all  --  anywhere             anywhere             /* cali:jWrgvDQ0xEZHmta3 */ /* Host endpoint policy accepted packet. */ mark match 0x10000/0x10000

相關內容