Azure 정책이 활성화된 Azure kubernetes를 설치했습니다. 나는 영사의 시작 안내서에 있는 단계를 그대로 따랐습니다.링크
그러나 consul이 배포되면 Pod가 배포되지 않습니다.
Replicaset을 확인해 보니 아래와 같은 오류가 발생했습니다.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 23s (x17 over 6m20s) replicaset-controller Error creating: admission webhook "validation.gatekeeper.sh" denied the request: [azurepolicy-psp-container-no-privilege-esc-30132221bc21e5b724da] Privilege escalation container is not allowed: controller
이 문제를 해결하는 방법은 무엇입니까?
출력이 포함된 세부 단계.
D:\consul_azure>git clone https://github.com/hashicorp/learn-consul-kubernetes.git
Cloning into 'learn-consul-kubernetes'...
remote: Enumerating objects: 504, done.
remote: Counting objects: 100% (504/504), done.
remote: Compressing objects: 100% (325/325), done.
remote: Total 504 (delta 260), reused 354 (delta 154), pack-reused 0
Receiving objects: 100% (504/504), 87.91 KiB | 173.00 KiB/s, done.
Resolving deltas: 100% (260/260), done.
D:\consul_azure>cd learn-consul-kubernetes/service-mesh/deploy
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>helm repo add hashicorp https://helm.releases.hashicorp.com
"hashicorp" already exists with the same configuration, skipping
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "aad-pod-identity" chart repository
...Successfully got an update from the "secrets-store-csi-driver" chart repository
...Successfully got an update from the "csi-secrets-store-provider-azure" chart repository
...Successfully got an update from the "hashicorp" chart repository
...Successfully got an update from the "spv-charts" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈Happy Helming!⎈
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>helm install -f config.yaml consul hashicorp/consul --version "0.32.1"
NAME: consul
LAST DEPLOYED: Mon Aug 16 12:36:55 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
Thank you for installing HashiCorp Consul!
Now that you have deployed Consul, you should look over the docs on using
Consul with Kubernetes available here:
https://www.consul.io/docs/platform/k8s/index.html
Your release is named consul.
To learn more about the release, run:
$ helm status consul
$ helm get all consul
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>helm status consul
NAME: consul
LAST DEPLOYED: Mon Aug 16 12:36:55 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
Thank you for installing HashiCorp Consul!
Now that you have deployed Consul, you should look over the docs on using
Consul with Kubernetes available here:
https://www.consul.io/docs/platform/k8s/index.html
Your release is named consul.
To learn more about the release, run:
$ helm status consul
$ helm get all consul
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>kubectl get pods --selector app=consul
No resources found in default namespace.
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>kubectl get pods
No resources found in default namespace.
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
consul-connect-injector-webhook-deployment 0/2 0 0 4m53s
consul-controller 0/1 0 0 4m53s
consul-webhook-cert-manager 0/1 0 0 4m53s
prometheus-server 0/1 0 0 4m53s
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>kubectl describe deploy consul-controller
Name: consul-controller
Namespace: default
CreationTimestamp: Mon, 16 Aug 2021 12:37:25 +0530
Labels: app=consul
app.kubernetes.io/managed-by=Helm
chart=consul-helm
component=controller
heritage=Helm
release=consul
Annotations: deployment.kubernetes.io/revision: 1
meta.helm.sh/release-name: consul
meta.helm.sh/release-namespace: default
Selector: app=consul,chart=consul-helm,component=controller,heritage=Helm,release=consul
Replicas: 1 desired | 0 updated | 0 total | 0 available | 1 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 25% max unavailable, 25% max surge
Pod Template:
Labels: app=consul
chart=consul-helm
component=controller
heritage=Helm
release=consul
Annotations: consul.hashicorp.com/connect-inject: false
Service Account: consul-controller
Containers:
controller:
Image: hashicorp/consul-k8s:0.26.0
Port: 9443/TCP
Host Port: 0/TCP
Command:
/bin/sh
-ec
consul-k8s controller \
-webhook-tls-cert-dir=/tmp/controller-webhook/certs \
-datacenter=dc1 \
-enable-leader-election \
-log-level="info" \
Limits:
cpu: 100m
memory: 50Mi
Requests:
cpu: 100m
memory: 50Mi
Environment:
HOST_IP: (v1:status.hostIP)
CONSUL_HTTP_ADDR: http://$(HOST_IP):8500
Mounts:
/tmp/controller-webhook/certs from cert (ro)
Volumes:
cert:
Type: Secret (a volume populated by a Secret)
SecretName: consul-controller-webhook-cert
Optional: false
Conditions:
Type Status Reason
---- ------ ------
Progressing True NewReplicaSetCreated
Available False MinimumReplicasUnavailable
ReplicaFailure True FailedCreate
OldReplicaSets: <none>
NewReplicaSet: consul-controller-dff49c9f4 (0/1 replicas created)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ScalingReplicaSet 5m9s deployment-controller Scaled up replica set consul-controller-dff49c9f4 to 1
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>kubectl get rs consul-controller-dff49c9f4
NAME DESIRED CURRENT READY AGE
consul-controller-dff49c9f4 1 0 0 5m30s
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>kubectl logs consul-controller-dff49c9f4
Error from server (NotFound): pods "consul-controller-dff49c9f4" not found
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>kubectl logs rs/consul-controller-dff49c9f4
error: timed out waiting for the condition
D:\consul_azure\learn-consul-kubernetes\service-mesh\deploy>kubectl describe rs/consul-controller-dff49c9f4
Name: consul-controller-dff49c9f4
Namespace: default
Selector: app=consul,chart=consul-helm,component=controller,heritage=Helm,pod-template-hash=dff49c9f4,release=consul
Labels: app=consul
chart=consul-helm
component=controller
heritage=Helm
pod-template-hash=dff49c9f4
release=consul
Annotations: deployment.kubernetes.io/desired-replicas: 1
deployment.kubernetes.io/max-replicas: 2
deployment.kubernetes.io/revision: 1
meta.helm.sh/release-name: consul
meta.helm.sh/release-namespace: default
Controlled By: Deployment/consul-controller
Replicas: 0 current / 1 desired
Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
Labels: app=consul
chart=consul-helm
component=controller
heritage=Helm
pod-template-hash=dff49c9f4
release=consul
Annotations: consul.hashicorp.com/connect-inject: false
Service Account: consul-controller
Containers:
controller:
Image: hashicorp/consul-k8s:0.26.0
Port: 9443/TCP
Host Port: 0/TCP
Command:
/bin/sh
-ec
consul-k8s controller \
-webhook-tls-cert-dir=/tmp/controller-webhook/certs \
-datacenter=dc1 \
-enable-leader-election \
-log-level="info" \
Limits:
cpu: 100m
memory: 50Mi
Requests:
cpu: 100m
memory: 50Mi
Environment:
HOST_IP: (v1:status.hostIP)
CONSUL_HTTP_ADDR: http://$(HOST_IP):8500
Mounts:
/tmp/controller-webhook/certs from cert (ro)
Volumes:
cert:
Type: Secret (a volume populated by a Secret)
SecretName: consul-controller-webhook-cert
Optional: false
Conditions:
Type Status Reason
---- ------ ------
ReplicaFailure True FailedCreate
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedCreate 23s (x17 over 6m20s) replicaset-controller Error creating: admission webhook "validation.gatekeeper.sh" denied the request: [azurepolicy-psp-container-no-privilege-esc-30132221bc21e5b724da] Privilege escalation container is not allowed: controller
답변1
Consul이 해결 방법으로 9단계에서 권한 에스컬레이션을 중단할 때까지정책 정의 할당, 정책 평가에서 Kubernetes 네임스페이스를 제외하려면 네임스페이스 제외 매개변수에 네임스페이스 목록을 지정합니다. kube-system, gatekeeper-system및 을 제외하는 것이 좋습니다 azure-arc.
특히,Kubernetes 클러스터는 컨테이너 권한 에스컬레이션을 허용해서는 안 됩니다.매개변수를 설정할 수 있습니다.네임스페이스 제외그리고컨테이너 제외, consul 네임스페이스 및/또는 컨테이너를 추가합니다.
에 따르면정책 정의:
"parameters": {
...
"excludedNamespaces": {
"type": "Array",
"metadata": {
"displayName": "Namespace exclusions",
"description": "List of Kubernetes namespaces to exclude from policy evaluation."
},
"defaultValue": [
"kube-system",
"gatekeeper-system",
"azure-arc"
]
},
...
"excludedContainers": {
"type": "Array",
"metadata": {
"displayName": "Containers exclusions",
"description": "The list of InitContainers and Containers to exclude from policy evaluation. The identify is the name of container. Use an empty list to apply this policy to all containers in all namespaces."
},
"defaultValue": []
},
...
}