2개의 VM 호스트에서 실행되는 K8s 클러스터가 있고, 6개의 k8s 노드와 3개의 마스터가 둘 다에 걸쳐 분할되어 있습니다. 최근 서버 중 하나가 다운되어 하나의 마스터와 3개의 노드만 남게 되는 문제가 발생했습니다. 모든 서비스가 여전히 실행 중이지만 API에 액세스할 수 없습니다. 나머지 마스터를 보면 kube API 서버 컨테이너가 자체적으로 등록할 수 없는 것처럼 보이기 때문에 나타나지 않습니다.
Api Server 컨테이너 로그는 다음과 같습니다.
I0831 09:25:10.464033 1 server.go:563] external host was not specified, using 192.168.50.51
I0831 09:25:10.464668 1 server.go:161] Version: v1.25.4
I0831 09:25:10.464708 1 server.go:163] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0831 09:25:11.015314 1 shared_informer.go:255] Waiting for caches to sync for node_authorizer
I0831 09:25:11.016581 1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0831 09:25:11.016605 1 plugins.go:161] Loaded 11 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
I0831 09:25:11.017954 1 plugins.go:158] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
I0831 09:25:11.017970 1 plugins.go:161] Loaded 11 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,CertificateSubjectRestriction,ValidatingAdmissionWebhook,ResourceQuota.
^[[A
W0831 09:25:27.246979 1 logging.go:59] [core] [Channel #3 SubChannel #5] grpc: addrConn.createTransport failed to connect to {
"Addr": "127.0.0.1:2379",
"ServerName": "127.0.0.1",
"Attributes": null,
"BalancerAttributes": null,
"Type": 0,
"Metadata": null
}. Err: connection error: desc = "transport: authentication handshake failed: read tcp 127.0.0.1:47046->127.0.0.1:2379: read: connection reset by peer"
W0831 09:25:27.246980 1 logging.go:59] [core] [Channel #4 SubChannel #6] grpc: addrConn.createTransport failed to connect to {
"Addr": "127.0.0.1:2379",
"ServerName": "127.0.0.1",
"Attributes": null,
"BalancerAttributes": null,
"Type": 0,
"Metadata": null
}. Err: connection error: desc = "transport: authentication handshake failed: read tcp 127.0.0.1:47040->127.0.0.1:2379: read: connection reset by peer"
W0831 09:25:27.247066 1 logging.go:59] [core] [Channel #1 SubChannel #2] grpc: addrConn.createTransport failed to connect to {
"Addr": "127.0.0.1:2379",
"ServerName": "127.0.0.1",
"Attributes": null,
"BalancerAttributes": null,
"Type": 0,
"Metadata": null
}. Err: connection error: desc = "transport: authentication handshake failed: read tcp 127.0.0.1:47036->127.0.0.1:2379: read: connection reset by peer"
E0831 09:25:31.020654 1 run.go:74] "command failed" err="context deadline exceeded"