Logs

Loki

# config.yaml
# ...
schema_config:
  configs:
    - from: 2022-10-01
      store: boltdb-shipper
      object_store: filesystem
      schema: v11
      index:
        prefix: index_
        period: 24h
# ...
table_manager:
  retention_deletes_enabled: true
  retention_period: 168h

# docker-compose.yaml
  loki:
    image: grafana/loki:2.3.0
    command: -config.file=/etc/loki/loki-config.yaml
    volumes:
      - ./loki/config.yaml:/etc/loki/loki-config.yaml
      - loki_data:/loki
    restart: unless-stopped
    expose:
      - 3100
    networks:
      - monitor-net
    labels:
      - "traefik.http.routers.loki.entrypoints=loki"
      - "traefik.http.routers.loki.rule=PathPrefix(`/`)"
      - "traefik.http.routers.loki.middlewares=auth"

Fluentd

Installation

Before installation - https://docs.fluentd.org/installation/before-install

https://docs.fluentd.org/installation/install-by-deb

# Ubuntu 18
curl -L https://toolbelt.treasuredata.com/sh/install-ubuntu-bionic-td-agent3.sh | sh
sudo systemctl start td-agent.service
sudo systemctl status td-agent.service

Файлы конфигурации сервиса находятся в директории /etc/td-agent

# --- testing
# console1
tail -f /var/log/td-agent/td-agent.log
# console2
$ curl -X POST -d 'json={"json":"message"}' http://localhost:8888/debug.test
# console1
# 2020-06-02 05:39:46.149078100 +0000 debug.test: {"json":"message"}

FluentBit

Installation

Daemon

https://docs.fluentbit.io/manual/ https://docs.fluentbit.io/manual/installation/linux/ubuntu

# 18.04
wget -qO - https://packages.fluentbit.io/fluentbit.key | sudo apt-key add -

# /etc/apt/sources.list
deb https://packages.fluentbit.io/ubuntu/bionic bionic main

sudo apt-get update
sudo apt-get install td-agent-bit
sudo  service td-agent-bit start
sudo  service td-agent-bit status
tail -f /var/log/syslog

Файлы конфигурации демона находятся в директории /etc/td-agent-bit

Client

sudo apt-get install cmake flex bison
# --- @see
#     https://fluentbit.io/download/
#     https://github.com/fluent/fluent-bit/releases
wget https://fluentbit.io/releases/1.4/fluent-bit-1.4.4.tar.gz
# --- testing
fluent-bit -i tail -p path=/var/log/syslog -o stdout

Configuration

Создаем файл конфигурации: забираем все логи из докера и отправляем в stdout

# fluent.conf 
[INPUT]
    Name tail
    Path /var/lib/docker/containers/*/*.log
    Parser docker

[FILTER]
    Name         nest
    Match        *
    Operation    lift
    Nested_under attrs

[FILTER]
    Name   grep
    Match  *
    Regex  location ^pro.itmonitoring.dev*

[OUTPUT]
    Name stdout
    Match *

# parser.conf
[PARSER]
    Name        docker
    Format      json
    Time_Key    time
    Time_Format %Y-%m-%dT%H:%M:%S

# используем sudo, т.к. по умолчанию доступа к директории с логами докера нет
sudo fluent-bit -c fluent.conf -R parser.conf

Fluent-Bit Forward Fluentd

Конфигурация для встречной работы:

fluentd

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match **>
  @type stdout
</match>

fluent-bit

[SERVICE]
...

[INPUT]
    Name tail
    storage.type  filesystem
    Path /var/lib/docker/containers/*/*.log
    Parser docker

[FILTER]
    Name         nest
    Match        *
    Operation    lift
    Nested_under attrs

[FILTER]
    Name   grep
    Match  *
    Regex  location ^pro.itmonitoring.dev*

[OUTPUT]
    Name          forward
    Match         *
    Host          127.0.0.1
    Port          24224

Logrotate

https://debianworld.ru/articles/rotaciya-logov-s-pomoshyu-logrotate-v-debian-ubuntu/

Создаем файл конфигурации для ротации

# sudo vim /etc/logrotate.d/ethereum-ruby-http
/tmp/ethereum_ruby_http.log {
	daily
	missingok
	rotate 7
	compress
	nodelaycompress
	notifempty
	su
}

Пример конфигурации для Rails-приложения

# /etc/logrotate.d/gosplan-api
/var/www/gosplan-api/current/log/*.log {
  daily
  missingok
  rotate 7
  compress
  delaycompress
  notifempty
  copytruncate
}

Тестовый запуск

sudo logrotate -d /etc/logrotate.d/gosplan-api

Запуск ротации

sudo logrotate -v -f /etc/logrotate.d/gosplan-api

Далее ротация будет запускаться автоматически в соответствии с файлом конфигурации.