Estoy intentando crear un clúster HA k3s usando Proxmox y una pequeña flota de Raspberriy PI 4B.
Para los PI todo funciona bien, pero al intentar instalar un maestro en una VM Proxmox no se inicia.
Mi configuración:
- Anfitrión: Proxmox 7.0.7 (también probé con 6.4.4)
- Invitado: Ubuntu 20.04.2
- K3S: v1.21.3+k3s1 (también probé con v1.19.13+k3s1)
- MaríaDB: 10.3
Estoy ejecutando estos comandos para instalar el maestro.
export K3S_DATASTORE_ENDPOINT='mysql://DB_USER:DB_PASSWORD@tcp(DB_IP:DB_PORT)/DB_SCHEME'
curl -sfL https://get.k3s.io | sh -s - server --node-taint CriticalAddonsOnly=true:NoExecute --tls-san NGINX_LOADBALANCER_IP
Este es el resultado de la instalación y el inicio:
[INFO] Using v1.19.13+k3s1 as release
[INFO] Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.19.1 3+k3s1/sha256sum-amd64.txt
[INFO] Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.19 .13+k3s1/k3s
[INFO] Verifying binary download
[INFO] Installing k3s to /usr/local/bin/k3s
[INFO] Creating /usr/local/bin/kubectl symlink to k3s
[INFO] Creating /usr/local/bin/crictl symlink to k3s
[INFO] Creating /usr/local/bin/ctr symlink to k3s
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3s
Job for k3s.service failed because the control process exited with error code.
See "systemctl status k3s.service" and "journalctl -xe" for details.
Revisé los registros:
systemctl status k3s.service
● k3s.service - Lightweight Kubernetes
Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Tue 2021-08-03 20:27:40 UTC; 2s ago
Docs: https://k3s.io
Process: 6181 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
Process: 6193 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
Process: 6194 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Process: 6195 ExecStart=/usr/local/bin/k3s server --node-taint CriticalAddonsOnly=true:NoExecute --tls-san NGINX_LOADBALANNCER_IP (code=exited, status=1/FAILURE)
Main PID: 6195 (code=exited, status=1/FAILURE)
Aug 03 20:27:40 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:27:40 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:27:40 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.
y:
journalctl -u k3s.service
Aug 03 20:06:17 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:17 k3svm1 sh[19450]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:17 k3svm1 sh[19451]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Aug 03 20:06:17 k3svm1 k3s[19460]: time="2021-08-03T20:06:17Z" level=info msg="Acquiring lock file /var/lib/rancher/k3s/data/.lock"
Aug 03 20:06:17 k3svm1 k3s[19460]: time="2021-08-03T20:06:17Z" level=info msg="Preparing data dir /var/lib/rancher/k3s/data/9df574741d2573cbbe6616e8624488b36b3340d077bc50da7cb167f1b08a64d1"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.748397535Z" level=info msg="Starting k3s v1.21.3+k3s1 (1d1f220f)"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.751745749Z" level=info msg="Configuring mysql database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.751876220Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.753348552Z" level=info msg="Database tables and indexes are up to date"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.757714719Z" level=info msg="Kine listening on unix://kine.sock"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.764631916Z" level=info msg="certificate CN=system:admin,O=system:masters signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.765377675Z" level=info msg="certificate CN=system:kube-controller-manager signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.766187231Z" level=info msg="certificate CN=system:kube-scheduler signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.766815165Z" level=info msg="certificate CN=kube-apiserver signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.767415198Z" level=info msg="certificate CN=system:kube-proxy signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 >
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.767950031Z" level=info msg="certificate CN=system:k3s-controller signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.768698847Z" level=info msg="certificate CN=k3s-cloud-controller-manager signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:0>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.769745716Z" level=info msg="certificate CN=kube-apiserver signed by CN=k3s-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.770870630Z" level=info msg="certificate CN=system:auth-proxy signed by CN=k3s-request-header-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:1>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.771882180Z" level=info msg="certificate CN=etcd-server signed by CN=etcd-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.772508382Z" level=info msg="certificate CN=etcd-client signed by CN=etcd-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.773399505Z" level=info msg="certificate CN=etcd-peer signed by CN=etcd-peer-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.813171353Z" level=info msg="certificate CN=k3s,O=k3s signed by CN=k3s-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.813556476Z" level=info msg="Active TLS secret (ver=) (count 9): map[listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn->
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.819862032Z" level=fatal msg="starting kubernetes: preparing server: bootstrap data already found and encrypted with different token"
Aug 03 20:06:18 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:06:18 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:06:18 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.
Aug 03 20:06:23 k3svm1 systemd[1]: k3s.service: Scheduled restart job, restart counter is at 1.
Aug 03 20:06:23 k3svm1 systemd[1]: Stopped Lightweight Kubernetes.
Aug 03 20:06:23 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:23 k3svm1 sh[19478]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:23 k3svm1 sh[19483]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.115279840Z" level=info msg="Starting k3s v1.21.3+k3s1 (1d1f220f)"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.119390931Z" level=info msg="Configuring mysql database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.119554649Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.121305745Z" level=info msg="Database tables and indexes are up to date"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.125898745Z" level=info msg="Kine listening on unix://kine.sock"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.146164308Z" level=fatal msg="starting kubernetes: preparing server: bootstrap data already found and encrypted with different token"
Aug 03 20:06:24 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:06:24 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:06:24 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.
Aug 03 20:06:29 k3svm1 systemd[1]: k3s.service: Scheduled restart job, restart counter is at 2.
Aug 03 20:06:29 k3svm1 systemd[1]: Stopped Lightweight Kubernetes.
Aug 03 20:06:29 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:29 k3svm1 sh[19507]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:29 k3svm1 sh[19508]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.565328025Z" level=info msg="Starting k3s v1.21.3+k3s1 (1d1f220f)"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.568959518Z" level=info msg="Configuring mysql database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.568994906Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.570693830Z" level=info msg="Database tables and indexes are up to date"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.575194321Z" level=info msg="Kine listening on unix://kine.sock"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.594809727Z" level=fatal msg="starting kubernetes: preparing server: bootstrap data already found and encrypted with different token"
Aug 03 20:06:29 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:06:29 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:06:29 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.
Aug 03 20:06:34 k3svm1 systemd[1]: k3s.service: Scheduled restart job, restart counter is at 3.
Aug 03 20:06:34 k3svm1 systemd[1]: Stopped Lightweight Kubernetes.
Aug 03 20:06:34 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:34 k3svm1 sh[19527]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:34 k3svm1 sh[19530]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
A partir de ahí continúan los reinicios.
Como mencioné, lo intenté con varias versiones, pero nada parece funcionar. Además, realmente no recibo ningún error. La única pista que encontré en diferentes problemas de GitHub fue habilitar la contenedorización con respecto a Raspberry PI editando /boot/cmdline.txt. Sin embargo, no recibo el problema en un PI sino en un Proxmox-VM.
¿Hay algo que este olvidando?
De alguna maneraeste chicoLogré que se ejecutara en la misma configuración. ¿Alguien más lo hizo funcionar y podría proporcionar alguna referencia?