kubernetes 클러스터, 단일 노드, debian 11을 설정했습니다. 그러나 CoreDNS가 아무것도 해결하지 못하는 것 같습니다. 포테이너가 리소스를 로드할 수 없어 이 문제가 발생했습니다.
http: proxy error: dial tcp: lookup kubernetes.default.svc on 10.96.0.10:53: read udp 10.244.0.4:57589->10.96.0.10:53: i/o timeout
이것이 내 DNS의 시간 초과임을 확인하고 서비스를 확인했습니다.
root@dmvandenberg:~/kubernetes# kubectl get svc -n kube-system -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 78m k8s-app=kube-dns
root@dmvandenberg:~/kubernetes# kubectl get pods --selector=k8s-app=kube-dns -o wide -n kube-system
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
coredns-78fcd69978-2b6cq 1/1 Running 0 79m 10.244.0.2 dmvandenberg.nl <none> <none>
coredns-78fcd69978-swprh 1/1 Running 0 79m 10.244.0.3 dmvandenberg.nl <none> <none>
다음 파일을 사용하여 클러스터를 설정했습니다.
cat init.sh init2.sh
kubeadm init --pod-network-cidr=10.244.0.0/16 --ignore-preflight-errors=all
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml
kubectl create -f https://docs.projectcalico.org/manifests/custom-resources.yaml
kubectl taint nodes --all node-role.kubernetes.io/master-
kubectl create -f localstorage.yml --save-config
kubectl create -f pvportainer.yml --save-config
kubectl patch storageclass local-storage -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
kubectl apply -n portainer -f https://raw.githubusercontent.com/portainer/k8s/master/deploy/manifests/portainer/portainer.yaml
나는 kubectl apply -f https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml또한 kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml kubectl create -f https://docs.projectcalico.org/manifests/custom-resources.yaml.
root@dmvandenberg:~/kubernetes# cat localstorage.yml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
root@dmvandenberg:~/kubernetes# cat pvportainer.yml
apiVersion: v1
kind: PersistentVolume
metadata:
name: portainer
spec:
capacity:
storage: 11Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: local-storage
local:
path: /dockerdirs/pvportainer
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- dmvandenberg.nl
다음 명령과 출력을 사용하여 문제를 DNS 확인으로 좁혔습니다.
root@dmvandenberg:~/kubernetes# kubectl logs --namespace=kube-system -l k8s-app=kube-dns -f & tcpdump -ani cni0 udp port 53
[5] 9505
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on cni0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
.:53
[INFO] plugin/reload: Running configuration MD5 = db32ca3650231d74073ff4cf814959a7
CoreDNS-1.8.4
linux/amd64, go1.16.4, 053c4d5
.:53
[INFO] plugin/reload: Running configuration MD5 = db32ca3650231d74073ff4cf814959a7
CoreDNS-1.8.4
linux/amd64, go1.16.4, 053c4d5
21:21:07.629395 IP 10.244.0.4.44224 > 10.244.0.2.53: 3488+ AAAA? kubernetes.default.svc.portainer.svc.cluster.local. (68)
21:21:07.629667 IP 10.244.0.4.43161 > 10.244.0.2.53: 433+ A? kubernetes.default.svc.portainer.svc.cluster.local. (68)
21:21:12.630395 IP 10.244.0.4.54508 > 10.244.0.3.53: 61466+ AAAA? kubernetes.default.svc.portainer.svc.cluster.local. (68)
21:21:12.630453 IP 10.244.0.4.46088 > 10.244.0.2.53: 55999+ A? kubernetes.default.svc.portainer.svc.cluster.local. (68)
^C
4 packets captured
4 packets received by filter
0 packets dropped by kernel
DNS 쿼리에 대한 응답을 볼 것으로 예상했지만 아무 것도 표시되지 않습니다. 인터넷에서 coredns의 코어 파일에 "log"를 추가하는 방법을 발견하여 이를 시도했지만 로그라인이 표시되지 않습니다. 이를 통해 tcpdump에 표시된 UDP 메시지가 coredns에 의해 읽혀지거나 수신되지 않는다는 것을 확신하게 되었습니다.
나는 모든 단계를 밟았다.https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-solution/, 그러나 이것은 더 이상 나를 이해하지 못했습니다.
하지만 이 이후에는 막히게 됩니다. 디버깅을 계속하려면 어떻게 해야 합니까? 무엇이 잘못되었을 수 있나요?
편집: 나는 이 가이드를 따라해 보았습니다.https://www.oueta.com/linux/create-a-debian-11-kubernetes-cluster-with-kubeadm/ 다른 인터페이스에서도 똑같은 결과가 나타납니다.
16:56:06.482769 cali6bd455d068f In IP 172.20.122.129.60650 > 10.96.0.10.53: 31215+ AAAA? kubernetes.default.svc.portainer.svc.cluster.local. (68)
16:56:06.482980 cali6bd455d068f In IP 172.20.122.129.35119 > 10.96.0.10.53: 8608+ A? kubernetes.default.svc.portainer.svc.cluster.local. (68)
16:56:11.483200 cali6bd455d068f In IP 172.20.122.129.57079 > 10.96.0.10.53: 61639+ AAAA? kubernetes.default.svc.portainer.svc.cluster.local. (68)
16:56:11.483309 cali6bd455d068f In IP 172.20.122.129.38249 > 10.96.0.10.53: 14976+ A? kubernetes.default.svc.portainer.svc.cluster.local. (68)
16:56:16.484367 cali6bd455d068f In IP 172.20.122.129.57768 > 10.96.0.10.53: 55396+ AAAA? kubernetes.default.svc.svc.cluster.local. (58)
16:56:16.484488 cali6bd455d068f In IP 172.20.122.129.53058 > 10.96.0.10.53: 50700+ A? kubernetes.default.svc.svc.cluster.local. (58)
16:56:21.484644 cali6bd455d068f In IP 172.20.122.129.58857 > 10.96.0.10.53: 18986+ AAAA? kubernetes.default.svc.svc.cluster.local. (58)
16:56:21.484702 cali6bd455d068f In IP 172.20.122.129.36861 > 10.96.0.10.53: 44020+ A? kubernetes.default.svc.svc.cluster.local. (58)
전체 인터페이스에서 tcpdump를 실행하면 다시 전송되는 ACK 메시지를 고려할 때 TCP가 작동하는 것으로 보입니다. 내가 알아차린 것은 10.96.0.10(서비스)에서 포드로 트래픽이 없다는 것인데, 그것이 필요한지 모르겠습니다.
17:03:29.224602 IP 169.254.167.173.36088 > 172.20.122.129.9000: Flags [.], ack 1, win 169, options [nop,nop,TS val 4014670766 ecr 4073454542], length 0
17:03:29.224869 IP 169.254.167.173.36088 > 172.20.122.129.9000: Flags [P.], seq 1:107, ack 1, win 169, options [nop,nop,TS val 4014670766 ecr 4073454542], length 106
17:03:29.224887 IP 172.20.122.129.9000 > 169.254.167.173.36088: Flags [.], ack 107, win 167, options [nop,nop,TS val 4073454542 ecr 4014670766], length 0
17:03:29.225273 IP 172.20.122.129.9000 > 169.254.167.173.36088: Flags [P.], seq 1:818, ack 107, win 167, options [nop,nop,TS val 4073454543 ecr 4014670766], length 817
17:03:29.225341 IP 169.254.167.173.36088 > 172.20.122.129.9000: Flags [.], ack 818, win 166, options [nop,nop,TS val 4014670767 ecr 4073454543], length 0
17:03:29.225399 IP 172.20.122.129.9000 > 169.254.167.173.36088: Flags [.], seq 818:7958, ack 107, win 167, options [nop,nop,TS val 4073454543 ecr 4014670767], length 7140
17:03:29.225422 IP 169.254.167.173.36088 > 172.20.122.129.9000: Flags [.], ack 7958, win 155, options [nop,nop,TS val 4014670767 ecr 4073454543], length 0
17:03:29.225430 IP 172.20.122.129.9000 > 169.254.167.173.36088: Flags [.], seq 7958:15098, ack 107, win 167, options [nop,nop,TS val 4073454543 ecr 4014670767], length 7140
17:03:29.225448 IP 169.254.167.173.36088 > 172.20.122.129.9000: Flags [.], ack 15098, win 138, options [nop,nop,TS val 4014670767 ecr 4073454543], length 0
17:03:29.225457 IP 172.20.122.129.9000 > 169.254.167.173.36088: Flags [P.], seq 15098:23486, ack 107, win 167, options [nop,nop,TS val 4073454543 ecr 4014670767], length 8388
17:03:29.225474 IP 169.254.167.173.36088 > 172.20.122.129.9000: Flags [.], ack 23486, win 119, options [nop,nop,TS val 4014670767 ecr 4073454543], length 0
17:03:29.225564 IP 172.20.122.129.9000 > 169.254.167.173.36088: Flags [F.], seq 23486, ack 107, win 167, options [nop,nop,TS val 4073454543 ecr 4014670767], length 0
17:03:29.225609 IP 169.254.167.173.36088 > 172.20.122.129.9000: Flags [R.], seq 107, ack 23486, win 166, options [nop,nop,TS val 4014670767 ecr 4073454543], length 0
17:03:29.524333 IP 172.20.122.129.9000 > 169.254.167.173.9984: Flags [.], ack 3370092883, win 166, options [nop,nop,TS val 4073454842 ecr 1976747960], length 0
17:03:29.524564 IP 169.254.167.173.9984 > 172.20.122.129.9000: Flags [.], ack 1, win 171, options [nop,nop,TS val 1976763065 ecr 4073424519], length 0
17:03:34.218598 IP 172.20.122.129.45239 > 10.96.0.10.53: 23854+ AAAA? kubernetes.default.svc. (40)
17:03:34.219065 IP 172.20.122.129.38604 > 10.96.0.10.53: 24098+ A? kubernetes.default.svc. (40)
17:03:34.388311 IP 172.20.122.129.9000 > 169.254.167.173.7394: Flags [.], ack 917, win 166, options [nop,nop,TS val 4073459706 ecr 1976752753], length 0
17:03:34.388402 IP 169.254.167.173.7394 > 172.20.122.129.9000: Flags [.], ack 1, win 171, options [nop,nop,TS val 1976767929 ecr 4073444530], length 0
17:03:34.388314 IP 172.20.122.129.9000 > 169.254.167.173.3949: Flags [.], ack 917, win 166, options [nop,nop,TS val 4073459706 ecr 1976752753], length 0
17:03:34.388424 IP 169.254.167.173.3949 > 172.20.122.129.9000: Flags [.], ack 1, win 171, options [nop,nop,TS val 1976767929 ecr 4073444530], length 0
17:03:34.388288 IP 172.20.122.129.9000 > 169.254.167.173.26855: Flags [.], ack 917, win 166, options [nop,nop,TS val 4073459706 ecr 1976752752], length 0
17:03:34.388544 IP 169.254.167.173.26855 > 172.20.122.129.9000: Flags [.], ack 1, win 171, options [nop,nop,TS val 1976767929 ecr 4073444529], length 0
17:03:39.216823 IP 169.254.167.173.36182 > 172.20.122.129.9000: Flags [S], seq 2192346809, win 43200, options [mss 1440,sackOK,TS val 4014680758 ecr 0,nop,wscale 8], length 0
17:03:39.216889 IP 172.20.122.129.9000 > 169.254.167.173.36182: Flags [S.], seq 1678785660, ack 2192346810, win 42840, options [mss 1440,sackOK,TS val 4073464535 ecr 4014680758,nop,wscale 8]
, length 0