Log Analyzer

From Segfault
Jump to: navigation, search


AWStats appears to be still maintained (last (beta) release from 2015-09-21), but the default layout of the stats is just awful. Let's see if we can change that.



GoAccess, a real-time log file analyzer. It's available as a Debian package too:

$ cat /etc/goaccess.conf
# Apache log date format
date-format %d/%b/%Y
# Common Log Format (CLF)
log-format %h %^[%d:%^] "%r" %s %b

$ sudo -u www-data goaccess -f /var/www/log/access.log


ModLogAn was my favourite log analyzer, but it's not maintained any more since 2013. Let's try to build it anyway :-)

Checkout the source, from CVS:

cvs -d:pserver:anonymous@modlogan.cvs.sourceforge.net:/cvsroot/modlogan login
cvs -z3 -d:pserver:anonymous@modlogan.cvs.sourceforge.net:/cvsroot/modlogan co -P modlogan
mv modlogan{,-cvs}

Obtaining a tarball from the last release (v0.8.13, 2004-09-04) wasn't so easy. The SourceForge site only had v0.8.10 (2003-09-19) but then there was modlogan.com:

> This page is intended as a place-holder for the modlogan manual and has been created 
> by Pipe Ten who use Modlogan extensively throughout their web hosting products.

Nice! However, they're not hosting the last version but referring to RepoForge instead:

wget http://pkgs.repoforge.org/modlogan/modlogan-0.8.13-1.2.rf.src.rpm
rpm2cpio modlogan-0.8.13-1.2.rf.src.rpm | cpio -ivd

We could also download from the internet archive (Archive):

wget http://web.archive.org/web/20061213203733/http://jan.kneschke.de/projects/modlogan/download/modlogan-0.8.13.tar.gz

Now we can extract the tarball:

$ md5sum modlogan*tar.gz
ba40c64f905a8d57edc3db5d9babfbfb  modlogan-0.8.13.tar.gz                    # This seems to be correct[1]
$ tar -xzf modlogan*tar.gz

Let's try to build it:

$ sudo apt-get install g++ libgd-dev libadns1-dev gettext
$ ./configure --prefix=/opt/modlogan
Configure finished:
xml-parser: expat
pcre      : 8.35
resolver  : enabled
graphics  : enabled
flow-tools: disabled                                                         # Installing flow-tools-dev didn't help :-\
localizer : enabled
io-wrapper: plain .gz

$ make && sudo make install

Before we can use it, we need to put some configuration files in place:

cd /opt/modlogan/etc/modlogan
for f in *dist; do sudo cp -pi $f ${f%%-dist}; done                          # Use defaults for now

Make some changes to modlogan.conf:

includepath    = /opt/modlogan/etc/modlogan
statedir       = /var/www/www.example.net/modlogan/state

hidereferrer   = "^http://www\.example\.net"

hostname       = www.example.net
outputdir      = /var/www/www.example.net/modlogan/

inputfile       = -                                                          # Let's use stdin for now

Create the missing directories:

sudo mkdir /var/www/www.example.net/modlogan/state
sudo chown -R www-data:www-data /var/www/www.example.net/modlogan/

And we should be ready to run:

$ sudo -u www-data /opt/modlogan/bin/modlogan -c /opt/modlogan/etc/modlogan/modlogan.conf < ~/access.log
writing month 12 - 2015
--> Setup       : Wall       0.02s, User       0.02s, System       0.00s <--
--> Parse       : Wall       0.00s, User       2.02s, System       0.83s <--
--> Process     : Wall       0.03s, User      10.08s, System       3.94s <--
--> Post-Process: Wall       0.27s, User       0.24s, System       0.03s <--
Throughput: 17511.16 rec/s (211885 records, 0 corrupt records, 0 skipped records, 0 ignored records)


TBD - https://piwik.org/log-analytics/


Webalizer seems to be dormant (last release from 2013-02-25)