我在 Kubernetes 叢集中設定了驗證服務，該叢集位於 Windows 11 Pro 電腦上的 Docker Desktop + WSL2 環境中。
例如，可以透過來自相同 Windows 電腦（WSL2 層之外）的 CURL 和 PostMan 請求來存取它。
然而，問題在於，無法從同一台 LAN 網路中的 Linux 電腦存取該其餘端點。
為了驗證與該身份驗證服務的連線是否確實有效，我設定了一個類型為「NodePort」的服務。透過從「kubectl get svc」獲得的外部端口，curl 請求在 Windows 和 Linux 電腦上都可以正常工作，並且服務會以正確的答案進行回應。
但請注意，NodePort 服務完全規避了 K8s 入口的 TSL/SSL，該入口帶有偽造的證書，因此該請求雖然成功，但僅適用於透過 HTTP 的請求。
因為我需要在 auth 服務的回應中使用 cookie，所以透過 HTTPS 的請求至關重要，但 K8s Ingress 根本不發送回應。
在windows機器上，我打開了相關連接埠80、443（和22只是為了確定），並在windows防火牆的連接規則中將linux機器添加到可信任的電腦中。我還嘗試暫時停用它，看看它是否確實阻止了連接嘗試。但是，該請求仍然被默默地丟棄（可能是由 Ingress）。
但在進行任何假設之前，讓我向您展示我的 LAN 的概述以及機器之間的 POST 請求，以及所有相關的 K8s yaml 檔案：
auth部署和服務定義如下：
apiVersion: apps/v1
kind: Deployment
metadata:
name: gts-auth-depl
spec:
replicas: 1
selector:
matchLabels:
app: gts-auth
template:
metadata:
labels:
app: gts-auth
spec:
containers:
- name: gts-auth
image: creativewarlock/gts-auth
imagePullPolicy: IfNotPresent
env:
- name: MONGO_URI
value: 'mongodb://gts-auth-mongo-srv:27017/auth'
- name: JWT_KEY
valueFrom:
secretKeyRef:
name: gts-jwt-secret
key: JWT_KEY
imagePullSecrets:
- name: docker-hub
---
apiVersion: v1
kind: Service
metadata:
name: gts-auth-srv
spec:
type: ClusterIP
selector:
app: gts-auth
ports:
- name: gts-auth
protocol: TCP
port: 3000
targetPort: 3000
身份驗證和 Web 用戶端的入口（雖然不相關，但只是為了顯示）：
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: gts-ingress-srv
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: 'true'
spec:
rules:
- host: gts.dev # Entry in System32/etc/hosts: 127.0.0.1 gts.dev (alternatively: localhost)
http:
paths:
- path: /api/users/?(.*) # will match /api/users/signin
pathType: Prefix
backend:
service:
name: gts-auth-srv
port:
number: 3000
- path: /?(.*) # map all other requests to web client
pathType: Prefix
backend:
service:
name: gts-client-srv
port:
number: 8080
用於外部/直接存取auth服務的NodePort服務：
apiVersion: v1
kind: Service
metadata:
name: gts-auth-srv-ext
spec:
type: NodePort
selector:
app: gts-auth
ports:
- name: gts-auth-srv
protocol: TCP
port: 3000
targetPort: 3000
Kubernetes 是這樣描述 Ingress 的：
k describe pod ingress-nginx-controller-6bf7bc7f94-krv2c -n ingress
-nginx
Name: ingress-nginx-controller-6bf7bc7f94-krv2c
Namespace: ingress-nginx
Priority: 0
Node: docker-desktop/192.168.65.4
Start Time: Thu, 11 Aug 2022 12:31:42 +0200
Labels: app.kubernetes.io/component=controller
app.kubernetes.io/instance=ingress-nginx
app.kubernetes.io/name=ingress-nginx
pod-template-hash=6bf7bc7f94
Annotations: <none>
Status: Running
IP: 10.1.28.235
IPs:
IP: 10.1.28.235
Controlled By: ReplicaSet/ingress-nginx-controller-6bf7bc7f94
Containers:
controller:
Container ID: docker://0cef9bec68d309011a32e18bf4f3a2a7ec7050b3db6cc7bafd87565e370f6695
Image: registry.k8s.io/ingress-nginx/controller:v1.3.0@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323ea1b5
Image ID: docker-pullable://registry.k8s.io/ingress-nginx/controller@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323ea1b5
Ports: 80/TCP, 443/TCP, 8443/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP
Args:
/nginx-ingress-controller
--publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
--election-id=ingress-controller-leader
--controller-class=k8s.io/ingress-nginx
--ingress-class=nginx
--configmap=$(POD_NAMESPACE)/ingress-nginx-controller
--validating-webhook=:8443
--validating-webhook-certificate=/usr/local/certificates/cert
--validating-webhook-key=/usr/local/certificates/key
State: Running
Started: Sun, 19 Feb 2023 11:50:07 +0100
Last State: Terminated
Reason: Error
Exit Code: 255
Started: Sat, 18 Feb 2023 11:44:18 +0100
Finished: Sun, 19 Feb 2023 11:49:36 +0100
Ready: True
Restart Count: 132
Requests:
cpu: 100m
memory: 90Mi
Liveness: http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=5
Readiness: http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
Environment:
POD_NAME: ingress-nginx-controller-6bf7bc7f94-krv2c (v1:metadata.name)
POD_NAMESPACE: ingress-nginx (v1:metadata.namespace)
LD_PRELOAD: /usr/local/lib/libmimalloc.so
Mounts:
/usr/local/certificates/ from webhook-cert (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-hpxhw (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
webhook-cert:
Type: Secret (a volume populated by a Secret)
SecretName: ingress-nginx-admission
Optional: false
kube-api-access-hpxhw:
ConfigMapOptional: <nil> DownwardAPI: true
QoS Class: Burstable
Node-Selectors: kubernetes.io/os=linux
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal RELOAD 54m (x3 over 95m) nginx-ingress-controller NGINX reload triggered due to a change in configuration
這是 Ingress 控制器的日誌。最後一行顯示了對 Windows 電腦的curl 請求的回應。
k logs -f ingress-nginx-controller-6bf7bc7f94-krv2c -n ingress-nginx
W0219 10:50:08.009613 7 client_config.go:617] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I0219 10:50:08.010796 7 main.go:230] "Creating API client" host="https://10.96.0.1:443"
W0219 10:50:30.162337 7 main.go:271] Initial connection to the Kubernetes API server was retried 1 times.
I0219 10:50:30.162386 7 main.go:274] "Running in Kubernetes cluster" major="1" minor="24" git="v1.24.1" state="clean" commit="3ddd0f45aa91e2f30c70734b175631bec5b5825a" platform="linux/amd64"
I0219 10:50:30.278395 7 main.go:104] "SSL fake certificate created" file="/etc/ingress-controller/ssl/default-fake-certificate.pem"
I0219 10:50:30.299891 7 ssl.go:531] "loading tls certificate" path="/usr/local/certificates/cert" key="/usr/local/certificates/key"
I0219 10:50:30.314983 7 nginx.go:258] "Starting NGINX Ingress controller"
I0219 10:50:30.321843 7 event.go:285] Event(v1.ObjectReference{Kind:"ConfigMap", Namespace:"ingress-nginx", Name:"ingress-nginx-controller", UID:"407f816e-98b9-4ab6-a08f-95efafa21ddc", APIVersion:"v1", ResourceVersion:"612", FieldPath:""}): type: 'Normal' reason: 'CREATE' ConfigMap ingress-nginx/ingress-nginx-controller
I0219 10:50:31.516611 7 nginx.go:301] "Starting NGINX process"
I0219 10:50:31.516698 7 leaderelection.go:248] attempting to acquire leader lease ingress-nginx/ingress-controller-leader...
I0219 10:50:31.517522 7 nginx.go:321] "Starting validation webhook" address=":8443" certPath="/usr/local/certificates/cert" keyPath="/usr/local/certificates/key"
I0219 10:50:31.517893 7 controller.go:167] "Configuration changes detected, backend reload required"
I0219 10:50:31.543621 7 leaderelection.go:258] successfully acquired lease ingress-nginx/ingress-controller-leader
I0219 10:50:31.543648 7 status.go:84] "New leader elected" identity="ingress-nginx-controller-6bf7bc7f94-krv2c"
I0219 10:50:31.610775 7 controller.go:184] "Backend successfully reloaded"
I0219 10:50:31.610910 7 controller.go:195] "Initial sync, sleeping for 1 second"
192.168.65.4 - - [19/Feb/2023:13:09:46 +0000] "POST /api/users/signin/ HTTP/1.1" 200 150 "-" "curl/7.83.1" 259 0.054 [default-gts-auth-srv-3000] [] 10.1.28.246:3000 150 0.050 200 5b8761d90f76b4c160b0a2effc980b85
來自我的 Linux 機器「Kaspar」的 curl HTTPS 請求未記錄在 Ingress pod 中，並且發送者從 NGINX 收到 404 HTML 文件：
Blacky@Kaspar:~$ curl -k -H 'Content-Type: application/json' https://192.168.0.57/api/users/signin/ -X POST -d '{ "email": "[email protected]", "password": "iHhRCF9Dm3wKT" }'
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
在 Windows 防火牆中，我已將 Linux 電腦新增至值得信賴的計算機，並透過 powershell 腳本明確設定這些規則：
$wsl_ip = (ubuntu.exe -c "ifconfig eth0 | grep 'inet '").trim().split()| where {$_}
$regex = [regex] "\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\b"
$ip_array = $regex.Matches($wsl_ip) | %{ $_.value }
$wsl_ip = $ip_array[0]
Write-Host "WSL Machine IP: ""$wsl_ip"""
New-NetFireWallRule -DisplayName 'WSL SSL' -Direction Outbound -LocalPort 443 -Action Allow -Protocol TCP
New-NetFireWallRule -DisplayName 'WSL SSL' -Direction Inbound -LocalPort 443 -Action Allow -Protocol TCP
#### ------------ Clear all portproxy rules ------------ ####
netsh int portproxy reset all
#### ------------ SSH rule - port 22 ------------ ####
netsh interface portproxy add v4tov4 listenport=22 listenaddress=0.0.0.0 connectport=22 connectaddress=$wsl_ip
#### ------------ Webserver SSL rule - port 443 ------------ ####
netsh interface portproxy add v4tov4 listenport=443 listenaddress=0.0.0.0 connectport=443 connectaddress=$wsl_ip
#### ------------ Webserver SSL rule - port 80 ------------ ####
netsh interface portproxy add v4tov4 listenport=80 listenaddress=0.0.0.0 connectport=80 connectaddress=$wsl_ip
所以我的主要問題是：
Kubernetes 中的（自簽名）假憑證是否造成了麻煩？我應該使用正確的 LetEncrypt 憑證在我的 LAN 中使用嗎？
由於curl/postman從Windows機器（WSL2層之外）請求工作，我首先解決的是配置Windows防火牆的問題，但K8s Ingress似乎沒有正確回應該請求。但是，我缺乏分析 Ingress 內部發生情況的工具。除了「kubectl messages -f <ingress...> -n ingress-nginx」之外，它們還有其他方式嗎？
非常歡迎任何有關如何從 LAN 內的另一台電腦取得 HTTPS 請求工作的建議。