![使用 iptables 將流量路由到 Kubernetes 集群](https://rvso.com/image/782618/%E4%BD%BF%E7%94%A8%20iptables%20%E5%B0%87%E6%B5%81%E9%87%8F%E8%B7%AF%E7%94%B1%E5%88%B0%20Kubernetes%20%E9%9B%86%E7%BE%A4.png)
我有一個 kubernetes 叢集(目前只有 1 個節點)在我的根伺服器 (netcup) 上運行,並嘗試將流量路由到它。我在網路方面很糟糕,所以希望有人可以幫助我。
在集群上金屬磅已安裝並將 calico 作為 CNI。 Metallb 負責為 K8s-LoadBalancers 分配和公佈 IP 位址,內部運作良好。所以我可以從伺服器內部獲得負載平衡的服務。我隻公開具有固定內部 IP 的單一服務(反向代理)
現在我想透過使用以下命令將流量直接從 Metallb 路由到 IP 位址,從而將其公開給外部iptables
iptables 中應用了以下規則:
iptables -A PREROUTING -t nat -p tcp -m tcp -j DNAT --to-destination <metallb-ip> -i eth0 --destination-port 80 -m comment --comment Redirect web traffic to cluster (80)
iptables -A PREROUTING -t nat -p tcp -m tcp -j DNAT --to-destination <metallb-ip> -i eth0 --destination-port 443 -m comment --comment Redirect web traffic to cluster (443)
iptables -A POSTROUTING -t nat -p tcp -m tcp -j SNAT --to-source <public-server-ip> -o eth0 --destination-port 80 -m comment --comment Redirect web traffic from cluster (80)
iptables -A POSTROUTING -t nat -p tcp -m tcp -j SNAT --to-source <public-server-ip> -o eth0 --destination-port 443 -m comment --comment Redirect web traffic from cluster (443)
(也許這些命令不是100%正確,因為我從我的ansible腳本中提取了它們。你可以在底部找到ansible部分)
據我了解,這應該使用 DNAT 將所有傳入 TCP 流量傳送到內部 IP,並且所有傳出流量將使用 SNAT 具有公用伺服器位址。那麼對於連接客戶端來說,他應該看起來像是直接與公共 IP/連接埠對話?
不幸的是,嘗試從 Internet 存取該服務會導致以下結果:
$> curl <public-server-ip>
curl: (7) Failed to connect to <public-server-ip> port 80 after 647 ms: Network is down
從伺服器存取有效:
$> curl <metallb-ip>
404 page not found # which the expected answer since its a blank Traefik reverse proxy
nmap 連接埠掃描目前會拋出以下錯誤:
$> nmap -Pn -p 80,443 <public-server-ip>
Starting Nmap 7.93 ( https://nmap.org ) at 2022-12-15 15:57 CET
Strange SO_ERROR from connection to <public-server-ip> (50 - 'Network is down') -- bailing scan
QUITTING!
但自從我透過 SSH 連接以來它就已啟動並運行。
目前沒有ufw
啟用。
我感謝任何幫助,因為正如我所說,我不太擅長網路。 ;)
安塞布爾腳本:
- name: DNAT port 80 to cluster
iptables:
table: nat
chain: PREROUTING
in_interface: eth0
protocol: tcp
match: tcp
jump: DNAT # REDIRECT
to_destination: <metallb-ip>
destination_port: 80
comment: Redirect web traffic to cluster (80)
become: yes
- name: DNAT port 443 to cluster
iptables:
table: nat
chain: PREROUTING
in_interface: eth0
protocol: tcp
match: tcp
destination_port: 443
jump: DNAT # REDIRECT
to_destination: <metallb-ip>
comment: Redirect web traffic to cluster (443)
become: yes
- name: SNAT port 80 from cluster
iptables:
table: nat
chain: POSTROUTING
out_interface: eth0
protocol: tcp
match: tcp
destination_port: 80
jump: SNAT # REDIRECT
to_source: <public-server-ip>
comment: Redirect web traffic from cluster (443)
become: yes
- name: SNAT port 443 from cluster
iptables:
table: nat
chain: POSTROUTING
out_interface: eth0
protocol: tcp
match: tcp
destination_port: 443
jump: SNAT # REDIRECT
to_source: <public-server-ip>
comment: Redirect web traffic from cluster (443)
become: yes
- name: Configure IP Masquerading
iptables:
table: nat
chain: POSTROUTING
out_interface: eth0
jump: MASQUERADE
更新: 在...的幫助下這文章我試著理解流程。新增了日誌記錄語句來調查哪裡出了問題。這就是我目前發現的:
為了檢查資料包遺失的位置,我新增了以下規則:
iptables -t raw -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[raw:PREROUTING]: "
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[mangle:PREROUTING]: "
iptables -t nat -A PREROUTING -p tcp --dport 80 -j LOG --log-prefix "[nat:PREROUTING]: "
iptables -t mangle -A POSTROUTING -p tcp --sport 80 -j LOG --log-prefix "[mangle:PREROUTING]: "
iptables -t nat -A POSTROUTING -p tcp --sport 80 -j LOG --log-prefix "[nat:PREROUTING]: "
/var/log/kern.log
在剛剛出現的raw
和mangle
log 語句下。所以看起來nat
-step 沒有被觸發。檢查 mangle 規則:
sudo iptables -t mangle -L
但無法辨識這裡的任何問題。
IP 轉送已啟用:
$> sudo sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1
那為什麼資料包沒有到達nat
步驟呢?
$> sudo iptables -t mangle -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
cali-PREROUTING all -- anywhere anywhere /* cali:6gwbT8clXdHdC1b1 */
LOG tcp -- anywhere anywhere tcp dpt:http LOG level warning prefix "[mangle:PREROUTING]: "
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
cali-POSTROUTING all -- anywhere anywhere /* cali:O3lYWMrLQYEMJtB5 */
Chain KUBE-IPTABLES-HINT (0 references)
target prot opt source destination
Chain KUBE-KUBELET-CANARY (0 references)
target prot opt source destination
Chain KUBE-PROXY-CANARY (0 references)
target prot opt source destination
Chain cali-from-host-endpoint (1 references)
target prot opt source destination
Chain cali-to-host-endpoint (1 references)
target prot opt source destination
Chain cali-PREROUTING (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere /* cali:6BJqBjBC7crtA-7- */ ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere /* cali:KX7AGNd6rMcDUai6 */ mark match 0x10000/0x10000
cali-from-host-endpoint all -- anywhere anywhere /* cali:wNH7KsA3ILKJBsY9 */
ACCEPT all -- anywhere anywhere /* cali:Cg96MgVuoPm7UMRo */ /* Host endpoint policy accepted packet. */ mark match 0x10000/0x10000
Chain cali-POSTROUTING (1 references)
target prot opt source destination
RETURN all -- anywhere anywhere /* cali:NX-7roTexQ3fGRfU */ mark match 0x10000/0x10000
MARK all -- anywhere anywhere /* cali:nnqPh8lh2VOogSzX */ MARK and 0xfff0ffff
cali-to-host-endpoint all -- anywhere anywhere /* cali:nquN8Jw8Tz72pcBW */ ctstate DNAT
RETURN all -- anywhere anywhere /* cali:jWrgvDQ0xEZHmta3 */ /* Host endpoint policy accepted packet. */ mark match 0x10000/0x10000