☰

Option 1

tag-icon • tag-icon docker prometheus ubuntu-22.04 snap
Option 1

Ich verwende einen Ubuntu-Server mit einem aus dem Server-Setup (Snap) installierten Docker-Dienst und Prometheus, das in einem Docker-Container ausgeführt wird.

Server:

Distributor ID: Ubuntu
Description:    Ubuntu 22.04.3 LTS
Release:        22.04
Codename:       jammy

Docker

sudo snap services
Service                          Startup  Current   Notes
docker.dockerd                   enabled  active    -
docker.nvidia-container-toolkit  enabled  inactive  -
lxd.activate                     enabled  inactive  -
lxd.daemon                       enabled  inactive  socket-activated
lxd.user-daemon                  enabled  inactive  socket-activated

Schnappschuss

snap    2.61.1
snapd   2.61.1
series  16
ubuntu  22.04
kernel  5.15.0-91-generic

Prometheus docker-compose.yml

version: '3.8'
volumes:
  prometheus_data: {}

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    restart: unless-stopped
    volumes:
      - /home/xxxx/data/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'

    extra_hosts:
      - "host.docker.internal:host-gateway"

    ports:
      - "9090:9090/tcp"

Docker-IP:

ip addr show docker0
    8: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
        link/ether 02:42:18:5c:09:7f brd ff:ff:ff:ff:ff:ff
        inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
           valid_lft forever preferred_lft forever
        inet6 fe80::42:18ff:fe5c:97f/64 scope link
           valid_lft forever preferred_lft forever

Ausgabe

Ich möchte also Metriken für meinen Prometheus-Server abrufen, der in einem Docker-Container ausgeführt wird. Ich verwende diese Dokumentation:https://docs.docker.com/config/daemon/prometheus/

Am Anfang heißt es:Fügen Sie die folgende Konfiguration hinzu:

Linux: /etc/docker/daemon.json

{
  "metrics-addr": "127.0.0.1:9323"
}

Ich habe es auch versucht mit:

{
   "metrics-addr": "172.17.0.1:9323"
}

Nach einigem Suchen habe ich folgende Frage/Antwort auf Askubuntu gefunden:https://askubuntu.com/questions/1296892/snap-edit-docker-configuration

Also habe ich die Datei daemon.json hinzugefügt zu

/var/snap/docker/current/etc/docker

und startete den Dienst neu mit:

sudo snap restart docker

nach dem Neustart erhalte ich immer noch keine Metriken vom Host. Ich teste dies mit:

curl http://localhost:9323/metrics
curl: (7) Failed to connect to localhost port 9323 after 0 ms: Connection refused
curl http://172.17.0.1:9323/metrics
curl: (7) Failed to connect to 172.17.0.1 port 9323 after 0 ms: Connection refused

Prometheus-Zielfehler

Docker-Systeminformationen

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.10.4)
  compose: Docker Compose (Docker Inc., v2.17.2)

Server:
 Containers: 20
  Running: 20
  Paused: 0
  Stopped: 0
 Images: 20
 Server Version: 20.10.24
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc version:
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.15.0-91-generic
 Operating System: Ubuntu Core 22
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 31.26GiB
 Name: blackholeport
 ID: AMKQ:XQSW:OPFY:ZLWE:VCVI:BKBB:NVO4:5WYJ:W6AZ:JB7M:7T4S:WXNC
 Docker Root Dir: /var/snap/docker/common/var-lib-docker
 Debug Mode: false
 Username: hagbart
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Docker-Inspektion Prometheus

[
    {
        "Id": "ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684",
        "Created": "2024-01-19T10:23:03.520072505Z",
        "Path": "/bin/prometheus",
        "Args": [
            "--config.file=/etc/prometheus/prometheus.yml"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 83237,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2024-01-19T13:00:24.320370241Z",
            "FinishedAt": "2024-01-19T13:00:17.400204943Z"
        },
        "Image": "sha256:2a72b385beafd2b3685ad337d14d23f6851f0ff335c5cd9a04c692b3eb846196",
        "ResolvConfPath": "/var/snap/docker/common/var-lib-docker/containers/ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684/resolv.conf",
        "HostnamePath": "/var/snap/docker/common/var-lib-docker/containers/ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684/hostname",
        "HostsPath": "/var/snap/docker/common/var-lib-docker/containers/ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684/hosts",
        "LogPath": "/var/snap/docker/common/var-lib-docker/containers/ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684/ded87ebe4dafa575a06781beacf7e33541a37d23c1594ce0ac80041929ba5684-json.log",
        "Name": "/prometheus",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "docker-default",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/home/lukas/data/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:rw"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "prometheus_default",
            "PortBindings": {
                "9090/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "9090"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "unless-stopped",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "CgroupnsMode": "private",
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": [
                "host.docker.internal:host-gateway"
            ],
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": null,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "Mounts": [
                {
                    "Type": "volume",
                    "Source": "prometheus_prometheus_data",
                    "Target": "/prometheus",
                    "VolumeOptions": {}
                }
            ],
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/snap/docker/common/var-lib-docker/overlay2/5656a88cbb1d0e0fd4ed939b0cdb30de86b7d2e4af2afd06d2ea33d8dd110199-init/diff:/var/snap/docker/common/var-lib-docker/overlay2/0aa7d62082adf6dcde04943d937cac5dae5d0609b12ae7319421cbf63131fca3/diff:/var/snap/docker/common/var-lib-docker/overlay2/59878a50f72e66a191110dadd15bf94d352cb1e0a543cf6ab0e96f210d9a16e5/diff:/var/snap/docker/common/var-lib-docker/overlay2/d7dd0384e6e888b48e94d3f32bf1e504c2ed1d11526ba5caba7a6cbaa43f1304/diff:/var/snap/docker/common/var-lib-docker/overlay2/8943641d04172c565f42e75963910fdf6df436bc60d3929284c657b16d831dd1/diff:/var/snap/docker/common/var-lib-docker/overlay2/6c943a9276d504534ec9c66ade625447ad516058dce64053b899283aac500395/diff:/var/snap/docker/common/var-lib-docker/overlay2/416d42bd54e76f8a2c78da2799ad6e6c5d2c3d78b822ed8e17ada9bdc0fc17a7/diff:/var/snap/docker/common/var-lib-docker/overlay2/89ab8fa214a8141179afa2c21071e565dc841d1e0f5a69dc372d5b6514456cb0/diff:/var/snap/docker/common/var-lib-docker/overlay2/d99204b78e6ddf1279d8f1b099318f921a61e9017d457955e5a53c5c2d17be8a/diff:/var/snap/docker/common/var-lib-docker/overlay2/259c1d3c5a275a0c51fe499c1a8a2857427cf592eb094dea65447b591ec66058/diff:/var/snap/docker/common/var-lib-docker/overlay2/721958698d3add899bd2597d3dab22267c3aaa40dc556eb7f0cdb8c564b14a74/diff:/var/snap/docker/common/var-lib-docker/overlay2/c122e772ebb24d66d663577b099800060d10e1d1fda0b38e75ea73bc2fdf6273/diff:/var/snap/docker/common/var-lib-docker/overlay2/b1a9df93abc1fbba6b8f3644eaf5ec9a76b8b2193ad1d934241242fb54b74cd0/diff",
                "MergedDir": "/var/snap/docker/common/var-lib-docker/overlay2/5656a88cbb1d0e0fd4ed939b0cdb30de86b7d2e4af2afd06d2ea33d8dd110199/merged",
                "UpperDir": "/var/snap/docker/common/var-lib-docker/overlay2/5656a88cbb1d0e0fd4ed939b0cdb30de86b7d2e4af2afd06d2ea33d8dd110199/diff",
                "WorkDir": "/var/snap/docker/common/var-lib-docker/overlay2/5656a88cbb1d0e0fd4ed939b0cdb30de86b7d2e4af2afd06d2ea33d8dd110199/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/home/lukas/data/config/prometheus/prometheus.yml",
                "Destination": "/etc/prometheus/prometheus.yml",
                "Mode": "rw",
                "RW": true,
                "Propagation": "rprivate"
            },
            {
                "Type": "volume",
                "Name": "prometheus_prometheus_data",
                "Source": "/var/snap/docker/common/var-lib-docker/volumes/prometheus_prometheus_data/_data",
                "Destination": "/prometheus",
                "Driver": "local",
                "Mode": "z",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "ded87ebe4daf",
            "Domainname": "",
            "User": "nobody",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "ExposedPorts": {
                "9090/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "--config.file=/etc/prometheus/prometheus.yml"
            ],
            "Image": "prom/prometheus:latest",
            "Volumes": {
                "/prometheus": {}
            },
            "WorkingDir": "/prometheus",
            "Entrypoint": [
                "/bin/prometheus"
            ],
            "OnBuild": null,
            "Labels": {
                "com.docker.compose.config-hash": "f387601dc597da679d6a2290f4192863d07c16791b9d33e29a4e84eb43b2e5df",
                "com.docker.compose.container-number": "1",
                "com.docker.compose.depends_on": "",
                "com.docker.compose.image": "sha256:2a72b385beafd2b3685ad337d14d23f6851f0ff335c5cd9a04c692b3eb846196",
                "com.docker.compose.oneoff": "False",
                "com.docker.compose.project": "prometheus",
                "com.docker.compose.project.config_files": "/home/lukas/BlackholePort/docker/prometheus/docker-compose.yml",
                "com.docker.compose.project.working_dir": "/home/lukas/BlackholePort/docker/prometheus",
                "com.docker.compose.service": "prometheus",
                "com.docker.compose.version": "2.17.2",
                "maintainer": "The Prometheus Authors <[email protected]>"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "b882dba1196b5ba11a81cedda028552f453ed82e42cec9fc8fc767d4192571f2",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "9090/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "9090"
                    },
                    {
                        "HostIp": "::",
                        "HostPort": "9090"
                    }
                ]
            },
            "SandboxKey": "/run/snap.docker/netns/b882dba1196b",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "prometheus_default": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "prometheus",
                        "prometheus",
                        "ded87ebe4daf"
                    ],
                    "NetworkID": "971e8873d46285b63d70fb6feeada747176a5427aa7a19127cb7e8dc8a7d8a37",
                    "EndpointID": "b67826c8e2736e5ac55ab67a8f9d67a6822dd3260a214622f65de968b0fa972e",
                    "Gateway": "192.168.32.1",
                    "IPAddress": "192.168.32.2",
                    "IPPrefixLen": 20,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:c0:a8:20:02",
                    "DriverOpts": null
                }
            }
        }
    }
]

Docker-Netzwerk ls

docker network ls
NETWORK ID     NAME                    DRIVER    SCOPE
f124d5db3e0b   bridge                  bridge    local
862535ef9371   deluge_default          bridge    local
86bf6736570f   foundryvtt_default      bridge    local
909b8162cdbe   grafana_default         bridge    local
299ce7db89b5   heimdall_default        bridge    local
92a81645a176   host                    host      local
a5e580aa1810   microbin_default        bridge    local
da8d9cb1d3e7   nginx-pm_default        bridge    local
055308848eba   none                    null      local
93afb48bbba2   o-portainer             bridge    local
460a87dcb670   o-security              bridge    local
ea16af5d1414   pihole_default          bridge    local
250440a093d7   portainer_default       bridge    local
971e8873d462   prometheus_default      bridge    local
dcbfb120b9b2   prometheus_monitoring   bridge    local
a482e951879d   prowlarr_default        bridge    local
9ab6f6734815   radarr_default          bridge    local
f6abd6de38d3   sonarr_default          bridge    local
0e062c280f80   spigotmc_default        bridge    local
640505be1145   teamspeak_default       bridge    local

nach dem Hinzufügen von Portainer zum Host-Netzwerkgleicher Fehler

docker-compose.yml

version: '3.8'
volumes:
  prometheus_data: {}

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    restart: unless-stopped
    volumes:
      - /home/lukas/data/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'

    extra_hosts:
      - "host.docker.internal:host-gateway"
    network_mode: "host"

    ports:
      - "9090:9090/tcp

Docker Inspect Prometheus (Netzwerkteil)

 "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "f114aceaadadca19fe9da85b04dbce4fc9aedae6d7f91a2bed8b3290d3186e82",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/run/snap.docker/netns/default",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "host": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "92a81645a17671cf440926ed83cdb6063790dcf2e18813722d96ccc90614ffb3",
                    "EndpointID": "4a4f2cef2556314c2feb526928b24784005a4617583e10813bae06f56d5db8e2",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "",
                    "DriverOpts": null
                }
            }
        }

Antwort1

Docker Compose erstellt ein Netzwerk namens „prometheus_default“ und hängt den Prometheus-Container an dieses Netzwerk an. Sie haben den Docker-Daemon so eingestellt, dass er Metriken auf der docker0Schnittstelle ankündigt, die sich im Allgemeinen auf dem bridgevon Docker erstellten Standardnetzwerk befindet.

Auf dem Host, auf dem Docker ausgeführt wird, können Sie überprüfen, ob der Docker-Daemon (-p, um Prozesse anzuzeigen) auf dem TCP-Metriken-Port (-t) mit der Nummer 9323 (-n) lauscht (-l) ssodernetstat

$ ss -tlpn | grep :9323
LISTEN 0      4096            127.0.0.1:9323       0.0.0.0:*    users:(("dockerd",pid=199176,fd=17))

Option 1

Versuchen Sie, diese Konfiguration zu docker-compose.yaml hinzuzufügen, um Prometheus zum Bridge-Netzwerk hinzuzufügen

version: '3.8'
volumes:
  prometheus_data: {}

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    restart: unless-stopped
    volumes:
      - /home/lukas/data/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'

    extra_hosts:
      - "host.docker.internal:host-gateway"
    network_mode: bridge

    ports:
      - "9090:9090/tcp

Stellen Sie dabei sicher, dass Docker so eingestellt ist, dass Metriken auf der Standardbrücke bekannt gegeben werden

{
   "metrics-addr": "172.17.0.1:9323"
}

Option 2

Ändern Sie den Docker-Daemon, um im prometheus_defaultNetzwerk zu lauschen

docker network inspect prometheus_default
[
    {
        "Name": "prometheus_default",
        "Id": "55a40edb42b2f3c1144236a765339e9b1c8dd0542df929430dc07829c5e156d5",
        "Created": "2023-03-31T16:51:48.212544059-04:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.19.0.0/16",
                    "Gateway": "172.19.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": true,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {},
        "Options": {},
        "Labels": {
            "com.docker.compose.network": "default",
            "com.docker.compose.project": "prometheus",
            "com.docker.compose.version": "1.29.2"
        }
    }
]

In daemon.json festlegen

{
   "metrics-addr": "172.19.0.1:9323"
}

Option 3

Erlauben Sie jedem Netzwerk den Zugriff auf Docker-Metriken (denken Sie daran, dass viel mehr Dienste Zugriff auf die Metrikschnittstelle des Docker-Daemons haben werden, was wünschenswert sein kann, aber nicht muss).

{
  "metrics-addr": "0.0.0.0:9323"
}

verwandte Informationen