chylex
/
Apache-Prometheus-Exporter
mirror of https://github.com/chylex/Apache-Prometheus-Exporter
1
0
Fork 0
Code
Prometheus exporter for Apache access logs. Processes multiple log files simultaneously (one file per virtual host).
25 Commits
2 Branches
0 Tags
168 KiB
Rust 99%
Dockerfile 1%
 
 
Go to file
chylex c2f47cf736
Skip existing lines when a log file is recreated due to spurious file creation events 		2023-10-28 11:47:21 +02:00
docker Use SIGINT stop signal in Docker Compose example 2023-09-30 23:40:04 +02:00
src Skip existing lines when a log file is recreated due to spurious file creation events 2023-10-28 11:47:21 +02:00
.dockerignore Add Docker Compose example 2022-12-15 16:24:45 +01:00
.gitattributes Add .gitattributes 2022-12-15 16:24:46 +01:00
.gitignore Add Rust project 2022-09-11 13:54:14 +02:00
Cargo.lock Release 1.0.0 2023-10-25 02:18:04 +02:00
Cargo.toml Release 1.0.0 2023-10-25 02:18:04 +02:00
Dockerfile Add Docker Compose example 2022-12-15 16:24:45 +01:00
LICENSE Initial commit 2022-09-10 21:28:56 +02:00
README.md Implement own log file tailing 2023-10-02 08:49:36 +02:00

README.md

Apache Prometheus Exporter

Exports Prometheus metrics from Apache access logs.

See the docker folder for an example setup using Docker Compose.

1. Configure Apache Access Log Format

The following snippet will create a log format named prometheus that includes all information the exporter expects. See Apache documentation for explanation of the format.

LogFormat "%t %h \"%r\" %>s %O %{ms}T \"%{Referer}i\" \"%{User-Agent}i\"" prometheus

2. Configure Apache Virtual Hosts

The following snippet is an example of how you could configure Apache to serve 3 domains from different folders using macros.

Each domain has its own access and error log file. The log files are rotated daily, with a dedicated folder for each day, and a ${APACHE_LOG_DIR}/latest/ folder with hard links to today's log files - this folder will be watched by the exporter.

<Macro Logs $domain>
	ErrorLog "|/usr/bin/rotatelogs -l -f -D -L ${APACHE_LOG_DIR}/latest/$domain.error.log ${APACHE_LOG_DIR}/%Y-%m-%d/$domain.error.log 86400"
	CustomLog "|/usr/bin/rotatelogs -l -f -D -L ${APACHE_LOG_DIR}/latest/$domain.access.log ${APACHE_LOG_DIR}/%Y-%m-%d/$domain.access.log 86400" prometheus
</Macro>

<Macro Domain $domain>
	<VirtualHost *:80>
		ServerName $domain
		DocumentRoot /var/www/html/$domain
		Use Logs $domain
	</VirtualHost>
</Macro>

Domain first.example.com
Domain second.example.com
Domain third.example.com

UndefMacro Domain
UndefMacro Logs

In this example, the first.example.com domain will be served from /var/www/html/first.example.com, and its logs will be written to:

  • ${APACHE_LOG_DIR}/latest/first.example.com.access.log
  • ${APACHE_LOG_DIR}/latest/first.example.com.error.log

3. Configure the Exporter

The exporter requires the following environment variables:

HTTP_HOST

The host that the HTTP server for metrics will listen on. If omitted, defaults to 127.0.0.1.

ACCESS_LOG_FILE_PATTERN, ERROR_LOG_FILE_PATTERN

The path to the access/error log files. You may use a single wildcard to match multiple files in a folder, or to match multiple folders in one level of the path. Whatever is matched by the wildcard will become the Prometheus label file. If there is no wildcard, the file label will be empty.

Example 1 (File Name Wildcard)

Log files for all domains are in /var/log/apache2/latest/ and are named <domain>.access.log and <domain>.error.log. This is the set up from the Apache configuration example above.

Pattern: /var/log/apache2/latest/*.access.log

  • Metrics for /var/log/apache2/latest/first.example.com.access.log will be labeled: first.example.com
  • Metrics for /var/log/apache2/latest/first.example.com.error.log will be labeled: first.example.com
  • Metrics for /var/log/apache2/latest/second.example.com.access.log will be labeled: second.example.com

The wildcard may appear anywhere in the file name.

Example 2 (Folder Wildcard)

Every domain has its own folder in /var/log/apache2/latest/ containing log files named access.log and error.log.

Pattern: /var/log/apache2/latest/*/access.log

  • Metrics for /var/log/apache2/latest/first.example.com/access.log will be labeled: first.example.com
  • Metrics for /var/log/apache2/latest/first.example.com/error.log will be labeled: first.example.com
  • Metrics for /var/log/apache2/latest/second.example.com/access.log will be labeled: second.example.com

The wildcard must not include any prefix or suffix, so /*/ is accepted, but /prefix_*/ or /*_suffix/ is not.

Notes

The exporter only searches for files when it starts. If you need the exporter to watch a new file or forget a deleted file, you must restart it.

4. Launch the Exporter

Start the exporter. The standard output will show which log files have been found, the web server host, and the metrics endpoint URL.

If no errors are shown, the exporter will begin reading the found log files from the end, and printing each line to the standard output. When a log file is rotated, the exporter will begin reading it from the beginning.

Press Ctrl-C to stop the exporter. Signals other than SIGINT are ignored.

Notes

The exporter is designed to work and tested with the rotatelogs tool in a Linux container. Any other tools or operating systems are unsupported.

If an error occurs while reading a file or re-opening a rotated file, the exporter will stop watching it and print the error to standard output.

5. Collect Prometheus Metrics

Currently, the exporter exposes only these metrics:

  • apache_requests_total total number of requests
  • apache_errors_total total number of errors

More detailed metrics will be added in the future.