Log Analyzer

AWStats

AWStats appears to be still maintained (last release from 2017-03-12), but the default layout of the stats is just awful. Let's see if we can change that.

TBD

GoAccess

GoAccess, a real-time log file analyzer. It's available as a Debian package too:

$ cat /etc/goaccess.conf
[...]
# Apache log date format
date-format %d/%b/%Y
# Common Log Format (CLF)
log-format %h %^[%d:%^] "%r" %s %b

$ sudo -u www-data goaccess -f /var/www/log/access.log

ModLogAn

ModLogAn was my favourite log analyzer, but it's not maintained any more since 2013. Let's try to build it anyway :-)

Checkout the source, from CVS:

cvs -d:pserver:anonymous@modlogan.cvs.sourceforge.net:/cvsroot/modlogan login
cvs -z3 -d:pserver:anonymous@modlogan.cvs.sourceforge.net:/cvsroot/modlogan co -P modlogan
mv modlogan{,-cvs}

Obtaining a tarball from the last release (v0.8.13, 2004-09-04) wasn't so easy. The SourceForge site only had v0.8.10 (2003-09-19) and www.modlogan.com (Archive) stated:

> As modlogan is nolonger under development it can be difficult to locate however it is
> currently carried in rpmforge / repoforge with version 0.8.13-1.2.el4.rf.

wget http://pkgs.repoforge.org/modlogan/modlogan-0.8.13-1.2.rf.src.rpm
rpm2cpio modlogan-0.8.13-1.2.rf.src.rpm | cpio -ivd

We could also download from the internet archive (Archive):

wget http://web.archive.org/web/20061213203733/http://jan.kneschke.de/projects/modlogan/download/modlogan-0.8.13.tar.gz

Now we can extract the tarball:

$ md5sum modlogan*tar.gz
ba40c64f905a8d57edc3db5d9babfbfb  modlogan-0.8.13.tar.gz              # This seems to be correct
$ tar -xzf modlogan*tar.gz

Let's try to build it:

$ sudo apt-get install g++ libgd-dev libadns1-dev gettext automake autoconf libtool
$ ./configure --prefix=/opt/modlogan
[...]
Configure finished:
xml-parser: expat
pcre      : 8.35
resolver  : enabled
graphics  : enabled
flow-tools: disabled                                                  # Installing flow-tools-dev didn't help :-\
localizer : enabled
io-wrapper: plain .gz

$ make && sudo make install

Before we can use it, we need to put some configuration files in place:

cd /opt/modlogan/etc/modlogan
for f in *dist; do sudo cp -pi $f ${f%%-dist}; done                   # Use defaults for now

Make some changes to modlogan.conf:

[global]
includepath    = /opt/modlogan/etc/modlogan
statedir       = /var/www/www.example.net/modlogan/state

[processor_web]
hidereferrer   = "^http://www\.example\.net"

[output_modlogan]
hostname       = www.example.net
outputdir      = /var/www/www.example.net/modlogan/

[input_clf]
inputfile       = -                                                   # Let's use stdin for now

Create the missing directories:

sudo mkdir /var/www/www.example.net/modlogan/state
sudo chown -R www-data:www-data /var/www/www.example.net/modlogan/

And we should be ready to run:

$ sudo -u www-data /opt/modlogan/bin/modlogan -c /opt/modlogan/etc/modlogan/modlogan.conf < ~/access.log
writing month 12 - 2015
--> Setup       : Wall       0.02s, User       0.02s, System       0.00s <--
--> Parse       : Wall       0.00s, User       2.02s, System       0.83s <--
--> Process     : Wall       0.03s, User      10.08s, System       3.94s <--
--> Post-Process: Wall       0.27s, User       0.24s, System       0.03s <--
Throughput: 17511.16 rec/s (211885 records, 0 corrupt records, 0 skipped records, 0 ignored records)

Matomo

Installation

Install from Git:

git clone --config filter.lfs.smudge=true https://github.com/matomo-org/matomo
cd matomo

Check out a stable branch:

git checkout -b local 3.13.1
git submodule update --init --recursive

Setup composer libraries:

curl -sS https://getcomposer.org/installer | php
php composer.phar install --no-dev

Install Matomo via the web interface. Once this is completed, we can disable some cruft:

./console development:disable
find -L . -type l -delete                                    # Why do we have broken symlinks?
chmod -c go-rwx *.md [A-Z]* .[a-z]*

Adjust ownerships and permissions:

sudo chown -cR www-data:www-data tmp/ config/ {matomo,piwik}.js
sudo chmod -cR a-w config/

Optionally, create a manifest.inc.php with matomo_manifest.sh:

sh matomo_manifest.sh `pwd`
sudo mv -iv manifest.inc.php.13534 config/manifest.inc.php

Updating

Update with:

cd /var/www/piwik
git checkout -b stable 3.13.2
git submodule update --init --recursive

php composer.phar install --no-dev
php console core:update --yes

Auto-Archiving

Enable Auto Archiving as a cronjob

00 * * * * www-data    cd /var/www/matomo && php console core:archive

Webalizer

Webalizer seems to be dormant, with its last release from 2013-08-26.