我創建了一個 K3s 多主嵌入式集群,如下所示:
主機名稱:k3s01
curl -sfL https://get.k3s.io | K3S_TOKEN=xxx INSTALL_K3S_EXEC="server --disable servicelb --disable traefik --bind-address=10.0.0.4 --tls-san 10.0.0.4 --node-external-ip=168.119.x.x --node-ip=10.0.0.4 --flannel-iface=enp7s0 --advertise-address=PUBIP-OF-LB --cluster-init" sh -
主機名稱:k8s02
curl -sfL https://get.k3s.io | K3S_TOKEN=xxx INSTALL_K3S_EXEC="server --disable servicelb --disable traefik --bind-address=10.0.0.2 --tls-san 10.0.0.2 --node-ip 10.0.0.2 --node-external-ip=168.119.x.x --flannel-iface=enp7s0 --server=https://10.0.0.4:6443" sh -
主機名稱:k8s03
curl -sfL https://get.k3s.io | K3S_TOKEN=xxx INSTALL_K3S_EXEC="server --disable servicelb --disable traefik --bind-address=10.0.0.3 --tls-san 10.0.0.3 --node-ip 10.0.0.3 --node-external-ip=168.119.x.x --flannel-iface=enp7s0 --server=https://10.0.0.4:6443" sh -
我可以透過 LB-IP 從本機與 kubectl 連線!負載平衡:TCP 6443 -> 6443
我還可以在上述任何節點中使用 kubectl。我為 Hetzner 部署了 CSI,效果也很好。透過他們的測試部署進行了測試!
然而,在所有這些之後(到目前為止工作正常)我嘗試安裝 ingress-nginx。部署開始沒有任何問題。但我發現與叢集內的 apiserver 通訊有問題,如 ingress-nginx-controller 的以下日誌所示:
E1204 11:42:25.216392 8 leaderelection.go:321] error retrieving resource lock ingress-nginx/ingress-controller-leader-nginx: Get "https://10.43.0.1:443/api/v1/namespaces/ingress-nginx/configmaps/ingress-controller-leader-nginx": dial tcp 10.43.0.1:443: connect: connection refused
嗯,奇怪!好的,讓我們做一些檢查:
kubectl get svc kubernetes -o yaml
apiVersion: v1
kind: Service
metadata:
creationTimestamp: "2020-12-04T11:22:25Z"
labels:
component: apiserver
provider: kubernetes
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:labels:
.: {}
f:component: {}
f:provider: {}
f:spec:
f:clusterIP: {}
f:ports:
.: {}
k:{"port":443,"protocol":"TCP"}:
.: {}
f:name: {}
f:port: {}
f:protocol: {}
f:targetPort: {}
f:sessionAffinity: {}
f:type: {}
manager: k3s
operation: Update
time: "2020-12-04T11:22:25Z"
name: kubernetes
namespace: default
resourceVersion: "10434"
selfLink: /api/v1/namespaces/default/services/kubernetes
uid: f0993556-3b7f-40aa-a293-45170cb03002
spec:
clusterIP: 10.43.0.1
ports:
- name: https
port: 443
protocol: TCP
targetPort: 6443
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
看起來gtm。
kubectl get endpoints -o yaml
apiVersion: v1
items:
- apiVersion: v1
kind: Endpoints
metadata:
creationTimestamp: "2020-12-04T11:22:25Z"
labels:
endpointslice.kubernetes.io/skip-mirror: "true"
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:labels:
.: {}
f:endpointslice.kubernetes.io/skip-mirror: {}
f:subsets: {}
manager: k3s
operation: Update
time: "2020-12-04T11:23:39Z"
name: kubernetes
namespace: default
resourceVersion: "808"
selfLink: /api/v1/namespaces/default/endpoints/kubernetes
uid: cb450392-b4c9-4c2f-bfde-1a3b20ac4b5d
subsets:
- addresses:
- ip: 167.233.x.x
- ip: 168.119.x.x
- ip: 168.119.x.x
ports:
- name: https
port: 6443
protocol: TCP
kind: List
metadata:
resourceVersion: ""
selfLink: ""
好的,為什麼 Pub IP 在這裡?讓我們從一個 Pod 檢查它以直接呼叫其中一個 IP:
kubectl exec -it ingress-controler-pod-xxxx -- bash
bash-5.0$ curl https://167.233.x.x:6443 --insecure
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "Unauthorized",
"reason": "Unauthorized",
"code": 401
}bash-5.0$ curl https://10.43.0.1:443
curl: (7) Failed to connect to 10.43.0.1 port 443: Connection refused
好吧..那很奇怪!
有時也會出現一些錯誤,例如:
Error from server: error dialing backend: dial tcp: lookup k8s02: Try again
當我嘗試將 top exex 放入 pod 或顯示日誌時出現。只有當我嘗試對另一台主機上執行的目標 Pod 執行此操作時,才會發生這種情況。
DNS有問題嗎?
cat /etc/resolv.conf
nameserver 127.0.0.53
options edns0 trust-ad
我無法透過主機名稱解析主機。但是我剛剛在 K3s 設定中指定了 IP。我的主機之間是否需要有效的 DNS?我的 K3s 安裝參數有問題嗎?
答案1
我遇到了類似的問題,是由於 DNS 解析配置錯誤引起的,請檢查是否可以相互解析節點主機名稱。