Project -> Implemented a monitoring and logging solution using Grafana, Loki, and Promtail to efficiently aggregate, visualize, and monitor logs
Grafana: Grafana is an open-source platform for monitoring and observability of infrastructure, applications, and services, allowing users to monitor the performance and health of their systems through customizable dashboards and visualizations.
Loki: It is responsible for storing logs and querying logs from all your applications and infrastructure. Loki integrates seamlessly with Grafana, allowing users to visualize and query logs in the same interface they use for metrics and alerts.
Promtail: Promtail is an agent that is used to collect logs and ship them to Loki. It is usually deployed to every machine that has applications to gather log data and forward it to a Loki instance for storage and querying.
Steps:
- Create an EC2 instance
- Install Grafana on Debian or Ubuntu
sudo apt-get update
sudo apt-get install grafana
sudo systemctl status grafana-server
Install Loki and Promtail using Docker
- Open port 3000 to run Grafana
Download Loki Config
wget https://raw.githubusercontent.com/grafana/loki/v2.8.0/cmd/loki/
loki-local-config.yaml -O loki-config.yaml
Run Loki Docker container
docker run -d --name loki -v $(pwd):/mnt/config -p 3100:3100 grafana/
loki:2.8.0 --config.file=/mnt/config/loki-config.yaml
Download Promtail Config
wget https://raw.githubusercontent.com/grafana/loki/v2.8.0/clients/cmd/
promtail/promtail-docker-config.yaml -O promtail-config.yaml
Run Promtail Docker container
docker run -d --name promtail -v $(pwd):/mnt/config -v /var/log:/var/log --link
loki grafana/promtail:2.8.0 --config.file=/mnt/config/promtail-config.yaml
- Open port 3100, as it is the default port that Loki listens on for incoming HTTP requests.
- Install ngnix to get logs of ngnix
- To know in which lines the word ngnix is used write ngnix
- Visual Dashboard to see how many times did the error occur.
