chylex/Apache-Prometheus-Exporter
1
0
Fork 0
mirror of https://github.com/chylex/Apache-Prometheus-Exporter.git synced 2024-11-21 05:42:45 +01:00
Code
Prometheus exporter for Apache access logs. Processes multiple log files simultaneously (one file per virtual host).
15 Commits 2 Branches 0 Tags 168 KiB
Rust 99%
Dockerfile 1%
alp
Go to file
chylex 589eaf4bc7
wip
2023-01-13 16:27:48 +01:00
docker Add support for watching error log files & add apache_errors_total metric 2023-01-07 17:27:25 +01:00
src wip 2023-01-13 16:27:48 +01:00
.dockerignore Add Docker Compose example 2022-12-15 16:24:45 +01:00
.gitattributes Add .gitattributes 2022-12-15 16:24:46 +01:00
.gitignore Add Rust project 2022-09-11 13:54:14 +02:00
Cargo.lock Add web server with metrics endpoint 2022-12-15 01:38:35 +01:00
Cargo.toml Add web server with metrics endpoint 2022-12-15 01:38:35 +01:00
Dockerfile Add Docker Compose example 2022-12-15 16:24:45 +01:00
LICENSE Initial commit 2022-09-10 21:28:56 +02:00
README.md Add warning to README about known issues 2023-01-08 11:51:15 +01:00

README.md

Apache Prometheus Exporter

Exports Prometheus metrics from Apache access logs.

See the docker folder for an example setup using Docker Compose.

1. Configure Apache Access Log Format

The following snippet will create a log format named prometheus that includes all information the exporter expects. See Apache documentation for explanation of the format.

LogFormat "%t %h \"%r\" %>s %O %{ms}T \"%{Referer}i\" \"%{User-Agent}i\"" prometheus

2. Configure Apache Virtual Hosts

The following snippet is an example of how you could configure Apache to serve 3 domains from different folders using macros.

Each domain has its own access and error log file. The log files are rotated daily, with a dedicated folder for each day, and a ${APACHE_LOG_DIR}/latest/ folder with hard links to today's log files - this folder will be watched by the exporter.

<Macro Logs $domain>
	ErrorLog "|/usr/bin/rotatelogs -l -f -D -L ${APACHE_LOG_DIR}/latest/$domain.error.log ${APACHE_LOG_DIR}/%Y-%m-%d/$domain.error.log 86400"
	CustomLog "|/usr/bin/rotatelogs -l -f -D -L ${APACHE_LOG_DIR}/latest/$domain.access.log ${APACHE_LOG_DIR}/%Y-%m-%d/$domain.access.log 86400" prometheus
</Macro>

<Macro Domain $domain>
	<VirtualHost *:80>
		ServerName $domain
		DocumentRoot /var/www/html/$domain
		Use Logs $domain
	</VirtualHost>
</Macro>

Domain first.example.com
Domain second.example.com
Domain third.example.com

UndefMacro Domain
UndefMacro Logs

In this example, the first.example.com domain will be served from /var/www/html/first.example.com, and its logs will be written to:

  • ${APACHE_LOG_DIR}/latest/first.example.com.access.log
  • ${APACHE_LOG_DIR}/latest/first.example.com.error.log

3. Configure the Exporter

The exporter requires the following environment variables:

HTTP_HOST

The host that the HTTP server for metrics will listen on. If omitted, defaults to 127.0.0.1.

ACCESS_LOG_FILE_PATTERN, ERROR_LOG_FILE_PATTERN

The path to the access/error log files. You may use a single wildcard to match multiple files in a folder, or to match multiple folders in one level of the path. Whatever is matched by the wildcard will become the Prometheus label file. If there is no wildcard, the file label will be empty.

Example 1 (File Name Wildcard)

Log files for all domains are in /var/log/apache2/latest/ and are named <domain>.access.log and <domain>.error.log. This is the set up from the Apache configuration example above.

Pattern: /var/log/apache2/latest/*.access.log

  • Metrics for /var/log/apache2/latest/first.example.com.access.log will be labeled: first.example.com
  • Metrics for /var/log/apache2/latest/first.example.com.error.log will be labeled: first.example.com
  • Metrics for /var/log/apache2/latest/second.example.com.access.log will be labeled: second.example.com

The wildcard may appear anywhere in the file name.

Example 2 (Folder Wildcard)

Every domain has its own folder in /var/log/apache2/latest/ containing log files named access.log and error.log.

Pattern: /var/log/apache2/latest/*/access.log

  • Metrics for /var/log/apache2/latest/first.example.com/access.log will be labeled: first.example.com
  • Metrics for /var/log/apache2/latest/first.example.com/error.log will be labeled: first.example.com
  • Metrics for /var/log/apache2/latest/second.example.com/access.log will be labeled: second.example.com

The wildcard must not include any prefix or suffix, so /*/ is accepted, but /prefix_*/ or /*_suffix/ is not.

Notes

At least one access log file and one error log file must be found when the exporter starts, otherwise the exporter immediately exits with an error.

If a log file is deleted, the exporter will automatically resume watching it if it is re-created later. If you want the exporter to forget about deleted log files, restart the exporter.

4. Launch the Exporter

Start the exporter. The standard output will show which log files have been found, the web server host, and the metrics endpoint URL.

Press Ctrl-C to stop the exporter.

Important: Due to library bugs, the exporter will currently not watch rotated log files. If you want to use this project right now, you will need to add the -c flag to rotatelogs, and restart the exporter after every rotation.

5. Collect Prometheus Metrics

Currently, the exporter exposes only these metrics:

  • apache_requests_total total number of requests
  • apache_errors_total total number of errors

More detailed metrics will be added in the future.