0010-monitoring

서버랑 컨테이너 상태 확인용.

Prometheus, Grafana, Alertmanager, Node Exporter, cAdvisor 구성.

cAdvisor는 Docker 컨테이너별 리소스 사용량을 Prometheus에 넘기기 위한 도구.

 

 

docker-compose.yml

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    restart: unless-stopped
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--storage.tsdb.retention.time=30d'
      - '--web.enable-lifecycle'
    volumes:
      - ../config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - ../config/prometheus/alert_rules.yml:/etc/prometheus/alert_rules.yml:ro
      - prometheus_data:/prometheus
    ports:
      - "9090:9090"
    environment:
      - TZ=Asia/Seoul
    networks:
      - monitoring

 

Prometheus는 메트릭 수집용.

 

 

Grafana.

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    restart: unless-stopped
    volumes:
      - grafana_data:/var/lib/grafana
      - ../config/grafana/provisioning:/etc/grafana/provisioning:ro
    environment:
      - TZ=Asia/Seoul
      - GF_SECURITY_ADMIN_USER=${GRAFANA_ADMIN_USER}
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD}
      - GF_USERS_ALLOW_SIGN_UP=false
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.grafana.rule=Host(`mon.yeonghoon.kim`)"
      - "traefik.http.routers.grafana.entrypoints=web"
      - "traefik.http.services.grafana.loadbalancer.server.port=3000"
    networks:
      - monitoring
      - proxy

 

 

Grafana는 mon.yeonghoon.kim으로 붙임.

 

 

Node Exporter.

  node-exporter:
    image: prom/node-exporter:latest
    container_name: node-exporter
    restart: unless-stopped
    command:
      - '--path.procfs=/host/proc'
      - '--path.sysfs=/host/sys'
      - '--path.rootfs=/rootfs'
      - '--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)'
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    ports:
      - "9100:9100"
    environment:
      - TZ=Asia/Seoul
    networks:
      - monitoring

 

 

호스트 CPU, 메모리, 디스크 확인용.

 

 

cAdvisor.

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    container_name: cadvisor
    restart: unless-stopped
    privileged: true
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk/:/dev/disk:ro
    ports:
      - "8080:8080"
    environment:
      - TZ=Asia/Seoul
    networks:
      - monitoring

 

 

컨테이너 메트릭 확인용.

 

 

prometheus.yml

global:
  scrape_interval: 15s
  evaluation_interval: 15s
  external_labels:
    monitor: 'docker-host'

alerting:
  alertmanagers:
    - static_configs:
        - targets:
            - alertmanager:9093

rule_files:
  - /etc/prometheus/alert_rules.yml

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'node-exporter'
    static_configs:
      - targets: ['node-exporter:9100']

  - job_name: 'cadvisor'
    static_configs:
      - targets: ['cadvisor:8080']

 

 

일단 서버 상태랑 Docker 컨테이너 상태를 볼 수 있게 구성.

알림 기준도 넣음.

CPU 80 / 90
메모리 85 / 95
디스크 90 / 95
컨테이너 다운