Set of tools used to generate usage statistics for 42l https://stats.42l.fr
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
neil 6b4ea9eb34 preventing the script to stop in case of goaccess error 1 month ago
scripts gen-stats: adding chown nobody:nogroup and removing the last log line 2 months ago
LICENSE Initial commit 2 years ago
README.md completely rewriting logs script 2 months ago
log2stats.sh preventing the script to stop in case of goaccess error 1 month ago

README.md

log2stats

Set of tools used to generate usage statistics for 42l.

Prerequisites

You need to use logrotate to cut logs perfectly, on a weekly basis. The delicate part is to avoid having some lines of logs that slips in the wrong week's logfiles.

log2stats.sh

Previously logs-rapports.py, log2stats.sh generates weekly and monthly GoAccess reports through multiple logfiles (one per service).

Each service has its own script to allow more granularity on the displayed panels.

For each service, the following reports are generated:

  • Report with crawlers ;
  • Report without crawlers ;
  • Report crawlers only ;
  • Internal report with specified parameters in scripts/internal.sh ;
  • Internal JSON report.

The internal reports are meant to display more sensitive information for system administration / monitoring purposes and are kept private.

On public reports, the sed command is used in scripts to hide one more octet from the visitor's IPs. The --anonymize-ip GoAccess parameter was hiding only one octet, which isn't enough.

The script must be run as root on the host, because it needs to create containers.

The script interrupts at the first error, so no further unwanted damage is done to your files.

Usability

The scripts aren't very modular, they've been made to answer specific needs and work in specific conditions that aren't always met for every infrastructure. Feel free to edit or improve them for your needs!