Define Prometheus
Limited Time Offer!
For Less Than the Cost of a Starbucks Coffee, Access All DevOpsSchool Videos on YouTube Unlimitedly.
Master DevOps, SRE, DevSecOps Skills!
What is Prometheus?
Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud. Since its inception in 2012, many companies and organizations have adopted Prometheus, and the project has a very active developer and user community. It is now a standalone open-source project and maintained independently of any company. To emphasize this independence and the open-source nature of Prometheus, the Cloud Native Computing Foundation (CNCF) accepted it as its second hosted project after Kubernetes in 2016.
Prometheus is designed to monitor the performance and health of various software services and applications by collecting metrics, storing them, and providing powerful querying capabilities.
Architecture of Prometheus
The architecture of Prometheus consists of several key components:
- Prometheus Server: The core component that scrapes and stores metrics.
- Pushgateway: For supporting short-lived jobs.
- Client Libraries: For instrumenting application code.
- Exporters: For services that cannot be instrumented directly.
- Alertmanager: For handling alerts.
- Prometheus Query Language (PromQL): For querying time series data.
Components of Prometheus
- Prometheus Server:
- Retrieves metrics from configured targets at specified intervals.
- Stores metrics in a time-series database.
- Allows querying of this data via PromQL.
- Client Libraries:
- Used to instrument application code to expose internal metrics via an HTTP endpoint.
- Official client libraries exist for Go, Java/Scala, Python, and Ruby.
- Exporters:
- Node Exporter: Exposes hardware and OS metrics.
- Blackbox Exporter: Allows blackbox probing of endpoints.
- Other Exporters: For databases, message queues, and various other services.
- Pushgateway:
- Allows ephemeral and batch jobs to push metrics to Prometheus.
- Alertmanager:
- Handles alerts generated by Prometheus rules.
- Manages alert dependencies and deduplication.
- Routes alerts to notification channels like email, PagerDuty, Slack, etc.
- Prometheus Web UI:
- Provides a web-based interface to view and query metrics.
Types of Rules
Prometheus supports several types of rules:
- Recording Rules:
- Precompute frequently needed or computationally expensive expressions and save their results as new time series.
- Example: “`yaml groups:
- name: example
rules: - record: job:http_inprogress_requests:sum
expr: sum(http_inprogress_requests) by (job)
“`
- name: example
- Alerting Rules:
- Define conditions under which an alert should be triggered.
- Example: “`yaml groups:
- name: example
rules: - alert: HighRequestLatency
expr: job:request_latency_seconds:mean5m{job=”myjob”} > 0.5
for: 10m
labels:
severity: page
annotations:
summary: High request latency
description: Request latency is above 0.5s for more than 10 minutes.
“`
- name: example
Top 10 Node Exporter PromQL Queries
Here are 10 commonly used PromQL queries for monitoring with Node Exporter:
- CPU Usage:
instance:node_cpu_utilisation:rate1m{instance="instance-name"} * 100
- Memory Usage:
node_memory_MemTotal_bytes - node_memory_MemFree_bytes - node_memory_Cached_bytes - node_memory_Buffers_bytes
- Disk Usage:
100 - ((node_filesystem_free_bytes{mountpoint="/"} / node_filesystem_size_bytes{mountpoint="/"}) * 100)
- Network Traffic:
rate(node_network_receive_bytes_total[5m])
- Load Average:
node_load1
- Number of Running Processes:
node_procs_running
- Disk I/O:
rate(node_disk_io_time_seconds_total[5m])
- Swap Usage:
node_memory_SwapTotal_bytes - node_memory_SwapFree_bytes
- File System Inodes:
node_filesystem_files{mountpoint="/"}
- Temperature:
promql node_hwmon_temp_celsius
References
- Prometheus Documentation
- Prometheus GitHub Repository
- Prometheus Exporters
- Prometheus Alerting Rules
- Prometheus Recording Rules
What is Prometheus?
Prometheus is an open-source systems monitoring and alerting tool. It is designed for reliability and to collect metrics from different services and applications, allowing developers and operations teams to monitor performance, identify issues, and ensure the stability of their systems.
Architecture of Prometheus?
Prometheus follows a pull-based approach for collecting metrics and consists of multiple components that work together to collect, store, query, and visualize metrics data. The architecture includes:
Components of Prometheus?
Prometheus Server
Client Libraries
Push Gateway
Exporters
Alertmanager
Web UI
Types of rules?
Record
Alert
Top 10 Node Exporter PromQL
CPU Usage
Memory usage
Disk usage
Network traffic
Server load
What is Prometheus?
Is an application used for event monitoring and alerting. It records metrics in a time series database built using an HTTP pull model, with flexible queries and real-time alerting
Architecture of Prometheus?
There is a Prometheus server pulling metrics from targets and saving it to storage. Then it can send data to Alertmanager to notify based on rules, and data can be explored using PromQL.
Components of Prometheus?
The main Prometheus server which scrapes and stores time series data
Client libraries for instrumenting application code
A push gateway for supporting short-lived jobs
Special-purpose exporters for services like HAProxy, StatsD, Graphite, etc.
An alertmanager to handle alerts
Various support tools
Types of rules?
alert and record
Top 10 Node Exporter PromQL
node_cpu_seconds_total
node_filesystem_avail_bytes
node_filesystem_size_bytes
node_network_receive_bytes_total
node_network_transmit_errs_total
node_network_transmit_packets_total
node_time_seconds
node_memory_MemFree_bytes
node_memory_Cached_bytes
node_memory_MemTotal_bytes
What is Prometheus?
Prometheus is an open-source monitoring solution written in Go that collects metrics data and stores that data in a time series database
Architecture of Prometheus?
The architecture of Prometheus consists of several key components that work together to collect, store, and process metrics data.
Components of Prometheus?
1. *Prometheus Server*: Responsible for scraping and storing metrics.
2. *Data Storage*: Stores the time-series data efficiently.
3. *PromQL*: The query language to extract metrics from the database.
4. *Service Discovery*: Mechanisms to discover target endpoints.
5. *Exporters*: Collects metrics from third-party services (e.g., Node Exporter for OS-level metrics).
6. *Alertmanager*: Manages alerts and routes notifications.
7. *Visualization Tools*: Such as Grafana for visualizing the metrics.
Types of rules?
Prometheus supports two types of rules which may be configured and then evaluated at regular intervals: recording rules and alerting rules.
Top 10 Node Exporter PromQL
1. *CPU Usage*:
promql
100 – (avg by (instance) (rate(node_cpu_seconds_total{mode=”idle”}[5m])) * 100)
2. *Memory Usage*:
promql
100 * (1 – (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes))
3. *Disk I/O Utilization*:
promql
rate(node_disk_io_time_seconds_total[5m]) * 100
4. *Disk Space Usage*:
promql
100 * (1 – node_filesystem_free_bytes / node_filesystem_size_bytes)
5. *Network Traffic*:
promql
rate(node_network_receive_bytes_total[5m]) + rate(node_network_transmit_bytes_total[5m])
6. *Load Average*:
promql
node_load1
7. *Swap Usage*:
promql
100 * (node_memory_SwapTotal_bytes – node_memory_SwapFree_bytes) / node_memory_SwapTotal_bytes
8. *System Uptime*:
promql
node_time_seconds – node_boot_time_seconds
9. *File Descriptors Usage*:
promql
node_filefd_allocated
10. *Temperature Monitoring*:
promql
node_hwmon_temp_celsius
1. Prometheus is an open-source system monitoring and alerting tool kits.
2. It consists of Prometheus Server, Metrics Collection, Data Model, Query Language (PromQL), Alerting, Storage, Visualization, Scalability and Federation.
3. The components of Prometheus are Prometheus Server, Metrics Collection, Time-Series Database, Data Model, PromQL (Prometheus Query Language), Alerting, Alert Manager, Service Discovery, Push Gateway, Visualization and Dashboards, Remote Storage and Federation
4. here are 2 main types of rules: Recording Rules and Alerting Rules.
5.
a) CPU Usage by Core:
100 – avg(irate(node_cpu_seconds_total{mode=”idle”}[5m])) by (cpu)
b) Memory Usage:
(1 – (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100
c) Disk Usage:
100 * (1 – (node_filesystem_avail_bytes{mountpoint=”/”,fstype=~”ext4|xfs”} / node_filesystem_size_bytes{mountpoint=”/”,fstype=~”ext4|xfs”}))
d) Load Average:
node_load1
e) Network Usage:
rate(node_network_receive_bytes_total{device!=”lo”}[5m])
f) Disk I/O Operations:
rate(node_disk_reads_completed_total{device=~”sda|nvme0n1″}[5m])
g) Process Count:
count(node_processes)
h) Uptime:
time() – node_time_seconds
i) Filesystem Usage:
100 * (1 – (node_filesystem_avail_bytes{mountpoint=”/data”} / node_filesystem_size_bytes{mountpoint=”/data”}))
j) Temperature (if available):
node_hwmon_temp_celsius{job=”node”}
What is Prometheus?
Prometheus is an open-source monitoring and alerting toolkit
Architecture of Prometheus?
Prometheus architecture consists of a multi-dimensional data model, a powerful query language (PromQL), and a pull-based data collection mechanism.
Types of rules?
Two main types, Recording and Alerting.
10 Node Exporter PromQL: CPU, Memory, DISK , IO, Network traffic, Load average, number of processes, uptime, swap space
What is Prometheus?
Prometheus is an open-source monitoring and alerting toolkit originally developed by SoundCloud. It is designed to record real-time metrics in a time-series database, providing powerful query capabilities and alerting mechanisms.
Architecture of PrometheusPrometheus has a multi-component architecture that includes:
Components of Prometheus
Types of RulesRecording Rules:
Used to precompute frequently needed or computationally expensive expressions and save their results as new time series.
Alerting Rules:
Define conditions under which an alert should be triggered and sent to the Alertmanager.
Top 10 Node Exporter PromQL Queries
What is Prometheus?
Its an opensource monitoring tool monitoring solution for collecting and aggregating metrics as time series data
Architecture of Prometheus?
Prometheus’ fundamental architecture is simple and centered on the server, which collects and stores time series data. The Prometheus server, a time-series database, a user interface, alerts, and several service discovery techniques are a few examples of the various parts that make up the overall architecture. For gathering metrics data, Prometheus uses a pull-based methodology. This differs from many other monitoring systems, which employ a push model. At certain intervals, Prometheus scrapes or grabs metrics from the applications rather than having the apps push their metrics to the system. By doing away with the necessity for application-side permissions, this paradigm streamlines the system’s management and security features. You can install and execute a single binary to use as Prometheus’ main server. The server executes tasks that scrape metric data from instrumented apps and localize it.
Components of Prometheus?
1 Prometheus Server
2 Data retrieval worker
3 Time Series DataBase(TSDB)
4 HTTP Server
5 Node Exporters
– Apache
– mysql
– haproxy
– windows
6 Push gateway
5 Alert Manager
Types of rules?
Recording rules and alerting rules
Recording rules: These rules are used for pre-calculating frequently used or computationally expensive queries. The results of recording rules are saved into their own time series.
Alerting rules: Alerting rules allow you to specify conditions based on Prometheus expression language expressions. When the alert expression results in one or more vector elements at a given point in time, the alert counts as active for these elements’ label sets. You can then send notifications about firing alerts to an external service. Alerting rules are based on PromQL queries
Top 10 Node Exporter PromQL
prometheus_http_requests_total
prometheus_http_requests_total{job=”prometheus”,group=”canary”}
prometheus_http_requests_total{environment=~”staging|testing|development”,method!=”GET”}
prometheus_http_requests_total{job=~”.*”} # Bad!
prometheus_http_requests_total{job=~”.+”} # Good!
prometheus_http_requests_total{job=~”.*”,method=”get”} # Good!
prometheus_http_requests_total[5m]
prometheus_http_requests_total{job=”prometheus”,group=”canary”}[2h]
prometheus_http_requests_total{environment=~”staging|testing|development”,method!=”GET”}[60m]
What is Prometheus?
# Prometheus is an open-source systems monitoring and alerting.
Architecture of Prometheus?
#Prometheus scrapes metrics from instrumented jobs, either directly or via an intermediary push gateway for short-lived jobs. It stores all scraped samples locally and runs rules over this data to either aggregate and record new time series from existing data or generate alerts. Grafana or other API consumers can be used to visualize the collected data.
Components of Prometheus?
#
-Prometheus server
-Targets and service discovery
-Metrics and Time series data
-Alert manager
-Client libraries and Exporters.
Types of rules ?Prometheus
#
counters
gauges
histograms
summaries
Top 10 Node Exporter PromQL Prometheus
#
node_cpu_seconds_total
node_memory_MemFree_bytes + node_memory_Cached_bytes
up{job=“prometheus”} or up{job=“node”}
histogram_quantile(0.9, rate(demo_api_request_duration_seconds_bucket[5m]))
deriv(demo_disk_usage_bytes[1h])
sum(node_filesystem_size_bytes)
rate(demo_api_request_duration_seconds_count[5m])
avg_over_time(go_goroutines[5m])
time()
absent(up{job=“some-job”})
rate(demo_api_request_duration_seconds_count[5m])[1h:]
Ad1.Prometheus it is a tool which allow us to monitoring infrastructure. This tool can works as container solution.
Ad2/3.This tool is consisting of graphana (to present data) alerta (to create notification), exporter (to takes data from clients), exporter
Ad4, Two types of rules: recording rules and alerting rules,
Please let me know where are all recording of the lessons we did. All lessons are good but there is a lot of materials.