K3s kann auf Proxmox VM nicht installiert werden

K3s kann auf Proxmox VM nicht installiert werden

Ich versuche, mit Proxmox und einer kleinen Flotte von Raspberry PIs 4B einen HA-k3s-Cluster zu erstellen.

Bei den PIs funktioniert alles einwandfrei, aber beim Versuch einen Master auf einer Proxmox-VM zu installieren, startet dieser nicht.

Mein Setup:

  • Host: Proxmox 7.0.7 (habe es auch mit 6.4.4 versucht)
  • Gast: Ubuntu 20.04.2
  • K3S: v1.21.3+k3s1 (ich habe es auch mit v1.19.13+k3s1 versucht)
  • MariaDB: 10.3

Ich führe diese Befehle aus, um den Master zu installieren

export K3S_DATASTORE_ENDPOINT='mysql://DB_USER:DB_PASSWORD@tcp(DB_IP:DB_PORT)/DB_SCHEME'

curl -sfL https://get.k3s.io | sh -s - server --node-taint CriticalAddonsOnly=true:NoExecute --tls-san NGINX_LOADBALANCER_IP

Dies ist die Ausgabe der Installation und des Starts:

[INFO]  Using v1.19.13+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.19.1                                                                                                                                                             3+k3s1/sha256sum-amd64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.19                                                                                                                                                             .13+k3s1/k3s
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Creating /usr/local/bin/ctr symlink to k3s
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO]  systemd: Starting k3s
Job for k3s.service failed because the control process exited with error code.
See "systemctl status k3s.service" and "journalctl -xe" for details.

Ich habe die Protokolle überprüft:

systemctl status k3s.service


● k3s.service - Lightweight Kubernetes
     Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Tue 2021-08-03 20:27:40 UTC; 2s ago
       Docs: https://k3s.io
    Process: 6181 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
    Process: 6193 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
    Process: 6194 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
    Process: 6195 ExecStart=/usr/local/bin/k3s server --node-taint CriticalAddonsOnly=true:NoExecute --tls-san NGINX_LOADBALANNCER_IP (code=exited, status=1/FAILURE)
   Main PID: 6195 (code=exited, status=1/FAILURE)

Aug 03 20:27:40 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:27:40 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:27:40 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.

Und:

journalctl -u k3s.service


Aug 03 20:06:17 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:17 k3svm1 sh[19450]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:17 k3svm1 sh[19451]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Aug 03 20:06:17 k3svm1 k3s[19460]: time="2021-08-03T20:06:17Z" level=info msg="Acquiring lock file /var/lib/rancher/k3s/data/.lock"
Aug 03 20:06:17 k3svm1 k3s[19460]: time="2021-08-03T20:06:17Z" level=info msg="Preparing data dir /var/lib/rancher/k3s/data/9df574741d2573cbbe6616e8624488b36b3340d077bc50da7cb167f1b08a64d1"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.748397535Z" level=info msg="Starting k3s v1.21.3+k3s1 (1d1f220f)"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.751745749Z" level=info msg="Configuring mysql database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.751876220Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.753348552Z" level=info msg="Database tables and indexes are up to date"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.757714719Z" level=info msg="Kine listening on unix://kine.sock"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.764631916Z" level=info msg="certificate CN=system:admin,O=system:masters signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.765377675Z" level=info msg="certificate CN=system:kube-controller-manager signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.766187231Z" level=info msg="certificate CN=system:kube-scheduler signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.766815165Z" level=info msg="certificate CN=kube-apiserver signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.767415198Z" level=info msg="certificate CN=system:kube-proxy signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 >
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.767950031Z" level=info msg="certificate CN=system:k3s-controller signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.768698847Z" level=info msg="certificate CN=k3s-cloud-controller-manager signed by CN=k3s-client-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:0>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.769745716Z" level=info msg="certificate CN=kube-apiserver signed by CN=k3s-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.770870630Z" level=info msg="certificate CN=system:auth-proxy signed by CN=k3s-request-header-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:1>
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.771882180Z" level=info msg="certificate CN=etcd-server signed by CN=etcd-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.772508382Z" level=info msg="certificate CN=etcd-client signed by CN=etcd-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.773399505Z" level=info msg="certificate CN=etcd-peer signed by CN=etcd-peer-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.813171353Z" level=info msg="certificate CN=k3s,O=k3s signed by CN=k3s-server-ca@1628021178: notBefore=2021-08-03 20:06:18 +0000 UTC notAfter=2022-08-03 20:06:18 +0000 UTC"
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.813556476Z" level=info msg="Active TLS secret  (ver=) (count 9): map[listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn->
Aug 03 20:06:18 k3svm1 k3s[19460]: time="2021-08-03T20:06:18.819862032Z" level=fatal msg="starting kubernetes: preparing server: bootstrap data already found and encrypted with different token"
Aug 03 20:06:18 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:06:18 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:06:18 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.
Aug 03 20:06:23 k3svm1 systemd[1]: k3s.service: Scheduled restart job, restart counter is at 1.
Aug 03 20:06:23 k3svm1 systemd[1]: Stopped Lightweight Kubernetes.
Aug 03 20:06:23 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:23 k3svm1 sh[19478]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:23 k3svm1 sh[19483]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.115279840Z" level=info msg="Starting k3s v1.21.3+k3s1 (1d1f220f)"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.119390931Z" level=info msg="Configuring mysql database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.119554649Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.121305745Z" level=info msg="Database tables and indexes are up to date"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.125898745Z" level=info msg="Kine listening on unix://kine.sock"
Aug 03 20:06:24 k3svm1 k3s[19489]: time="2021-08-03T20:06:24.146164308Z" level=fatal msg="starting kubernetes: preparing server: bootstrap data already found and encrypted with different token"
Aug 03 20:06:24 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:06:24 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:06:24 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.
Aug 03 20:06:29 k3svm1 systemd[1]: k3s.service: Scheduled restart job, restart counter is at 2.
Aug 03 20:06:29 k3svm1 systemd[1]: Stopped Lightweight Kubernetes.
Aug 03 20:06:29 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:29 k3svm1 sh[19507]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:29 k3svm1 sh[19508]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.565328025Z" level=info msg="Starting k3s v1.21.3+k3s1 (1d1f220f)"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.568959518Z" level=info msg="Configuring mysql database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.568994906Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.570693830Z" level=info msg="Database tables and indexes are up to date"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.575194321Z" level=info msg="Kine listening on unix://kine.sock"
Aug 03 20:06:29 k3svm1 k3s[19511]: time="2021-08-03T20:06:29.594809727Z" level=fatal msg="starting kubernetes: preparing server: bootstrap data already found and encrypted with different token"
Aug 03 20:06:29 k3svm1 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 03 20:06:29 k3svm1 systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 03 20:06:29 k3svm1 systemd[1]: Failed to start Lightweight Kubernetes.
Aug 03 20:06:34 k3svm1 systemd[1]: k3s.service: Scheduled restart job, restart counter is at 3.
Aug 03 20:06:34 k3svm1 systemd[1]: Stopped Lightweight Kubernetes.
Aug 03 20:06:34 k3svm1 systemd[1]: Starting Lightweight Kubernetes...
Aug 03 20:06:34 k3svm1 sh[19527]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Aug 03 20:06:34 k3svm1 sh[19530]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory

Von dort aus werden die Neustarts fortgesetzt.

Wie erwähnt habe ich es mit mehreren Versionen versucht, aber nichts scheint zu funktionieren. Außerdem bekomme ich nicht wirklich einen Fehler. Der einzige Hinweis, den ich in verschiedenen GitHub-Problemen gefunden habe, war, die Containerisierung für Raspberry PI zu aktivieren, indem ich /boot/cmdline.txt bearbeite. Allerdings bekomme ich das Problem nicht auf einem PI, sondern auf einer Proxmox-VM.

Übersehe ich etwas?

Irgendwiedieser Typhabe es geschafft, es im gleichen Setup zum Laufen zu bringen. Hat es jemand anderes zum Laufen gebracht und kann vielleicht eine Referenz liefern?

verwandte Informationen