I use AWStats to generate reports from my Apache webserver logs. Now, I prefer to use AWStats to generate statically linked reports, so instead of having the “Update now” option on the AWStats report page, I have the following script that runs every hour or so as a cron job to update the reports instead.
#!/bin/bash perl /path/to/awstats_buildstaticpages.pl -config=MYSITE -update -staticlinks -awstatsprog=/path/to/awstats.pl -dir=/var/www/MYSITE/statistics/ > /dev/null chown MYUSERNAME:MYGROUPNAME /var/www/MYSITE/statistics/awstats.MYSITE.*
The result of that little script is that AWStats prepares a detailed report of webserver activity for the current month. Now, that’s great but how can you maintain an archive of detailed reports for previous months? Answer; write another bash script…
Now, this script is somewhat more complicated than the two-line effort above but it will prepare a set of detailed reports for the previous month, store them in a date-named subfolder and generate a web page listing all sets of monthly reports it’s created so far. Just run it as a cron job on the first of every month and make sure it doesn’t conflict with the running of the regular update script above, especially if you’ve got AWStats’ file-lock option turned on.
NOTE: This script won’t go back through your whole history of apache logs, generating AWStats reports for each month of activity it can find. That’s a job for another script…
Here’s a directory tree by way of explanation…
# Website root /var/www/MYSITE/index.html # AWStats detailed report for current month, generated by the two-line script above /var/www/MYSITE/statistics/awstats.MYSITE.html # AWStats detailed report for October, created by script below when it ran on 1st November /var/www/MYSITE/statistics/2011_10/awstats.MYSITE.html # AWStats detailed report for September, created by script below when it ran on 1st October /var/www/MYSITE/statistics/2011_09/awstats.MYSITE.html # Web page created by script below with links to... # - Current month's detailed report # - Detailed report for October # - Detailed report for September /var/www/MYSITE/statistics/index.html
Below is the script itself. It won’t work out of the box – there are a few variables to set first, mostly paths to AWStats and your webserver. You can copy and paste it from this page or download the script here. As with the other scripts I’ve made available to download, change the extension and permissions before running it. There’s not much more to say about the script here as I’ve made extensive notes in the file. I hope you find it useful!
#!/bin/bash
## ---------------------------------------------------------------------------
# awstats_monthly_reports.sh
## ---------------------------------------------------------------------------
# Script to generate a full set of statically linked awstats reports
# for the previous calendar month for an individual website.
#
# Author: Richard Leszczynski
# Email: contact NOSPAM @makerdyne.com
# Created: 30/10/2011
# Modified: 26/11/2011
## ---------------------------------------------------------------------------
#
# USAGE:
# To be run once a month at the beginning of the month.
# User running the script must have execute permission for the awstats# perl
#scripts and write permissions for the directory in which the# statistics
# report is to be created. (so 'root', most likely)
#
# BEHAVIOUR:
# 1. Creates (if necessary) a 'statistics' subfolder for the website to
# store the monthly statistics reports.
# 2. Creates a subfolder within 'statistics' for the monthly report that
# is to be created. Folder name will be a numeric# representation of the year
# and month (e.g. 2011_10)
# 3. Creates a full set of statically-linked awstats reports# for the
# previous month using awstats_buildstaticpages.pl
# 4. Changes the owner & group of the folder/reports created; ideally to
# those of a regular user, not root.
# 5. Updates statistics/index.html with a link to the newly# created
# report
#
#
# REQUIREMENTS:
# awstats already configured for the website that the monthly# report is
# to be generated for. For instructions. see
# http://awstats.sourceforge.net/docs/awstats_setup.html
#
# RECOMMENDATIONS:
# That an awstats.pl command such as
# perl /PATH/TO/awstats.pl -config=MYSITE -update -output -staticlinks > /PATH/TO/mysite/statistics/awstats.MYSITE.html
# is already running on a regular basis through a cron job in order# to give
# you something to look at between the generation of each monthly report.
#
# If you change the path of the report that the command above outputs
# to the statistics directory, it will be linked to within the generated
#statistics/index.html page
#
# It is also recommended that an awstats.pl command such as
# perl /PATH/TO/awstats.pl -config=MYSITE -update
# is added as a prerotate action within /etc/logrotate.d/apache2. See the
# awstats documentation for more information on this.
## ---------------------------------------------------------------------------
# Paths & Variables Required: Change to suit your installation...
## ---------------------------------------------------------------------------
# Full path of your website's DocumentRoot - NO TRAILING SLASH!
# e.g. /var/www/MYSITE/
WEBSITE_DIR="/var/www/MYSITE"
# Full path of awstats.pl script
AWSTATS_PATH="/path/to/awstats.pl"
# Full path of awstats_buildstaticpages.sh
AWSTATS_BSP_PATH="/path/to/awstats_buildstaticpages.pl"
# awstats website configuration name. The text you'd enter in place
# of MYSITE in the command 'perl awstats.pl -config=MYSITE -update'
MYSITE="mywebsitename"
# User and group to own the created monthly report
# e.g. USER=myusername , GROUP=users , or
# e.g. USER=www-data , GROUP=www-data
MYUSER=myusername
MYGROUP=mygroupname
## ---------------------------------------------------------------------------
# Check for pre-existing statistics directory
WEBSITE_DIR=${WEBSITE_DIR%/}
if [ ! -d "$WEBSITE_DIR" ]; then
echo "$WEBSITE_DIR does not exist! Check path for WEBSITE_DIR has been set correctly. Exiting..."
exit -1
fi
STATS_DIR=$WEBSITE_DIR/statistics
if [ ! -d "$STATS_DIR" ]; then
mkdir "$STATS_DIR"
fi
chown $MYUSER:$MYGROUP "$STATS_DIR"
# New Subdirectory to hold the new set of monthly reports
LMONTH=$(date --date="last month" +%m)
LYEAR=$(date --date="last month" +%Y)
REPORT_DIR=("$STATS_DIR"/"$LYEAR"_"$LMONTH")
if [ ! -d "$REPORT_DIR" ]; then
mkdir "$REPORT_DIR"
fi
chown $MYUSER:$MYGROUP "$REPORT_DIR"
# Run awstats_buildstaticpages.pl to generate a complete set of
# statically linked reports for the last month
perl "$AWSTATS_BSP_PATH" -config=$MYSITE -update -month=$LMONTH -year=$LYEAR -staticlinks -awstatsprog="$AWSTATS_PATH" -dir="$REPORT_DIR" >/dev/null
chown $MYUSER:$MYGROUP "$REPORT_DIR/awstats.$MYSITE".*
# Create an index.html file to link to both the current/rolling report
# and a history of all the monthly reports created to date
INDEX_FILE=$STATS_DIR/index.html
echo "<!--File generated automatically by awstats_monthly_reports.sh-->" > "$INDEX_FILE"
echo "
<h1>AWSTATS Website Statistics for $MYSITE</h1>
" >> "$INDEX_FILE"
echo "
<h2>Rolling report for this month</h2>
" >> "$INDEX_FILE"
echo "<a href="awstats.$MYSITE.html">1st $(date --date="today" +%B %Y) to today</a>" >> "$INDEX_FILE"
echo "
<h2>Archive of monthly website activity reports</h2>
" >> "$INDEX_FILE"
# obtain a list of directories within the STATS_DIR that match the date format which the
# monthly repots are stored in
cd "$STATS_DIR"
MONTHLY_DIRS=$(ls -rd */ | grep -E ^[12][9012][0-9]{2}_[01][0-9]/)
for dr in $MONTHLY_DIRS
do
TEMP_YEAR=${dr:0:4}
TEMP_MONTH=${dr:5:2}
TEMP_MONTH=$(echo $TEMP_MONTH | sed 's/^0*//')
(( MONTH_OFFSET = $(date --date="today" +%-m) - "$TEMP_MONTH" ))
echo "<a>$(date --date="$MONTH_OFFSET months ago" +%B) $TEMP_YEAR</a>
" >> "$INDEX_FILE"
done
echo "" >> "$INDEX_FILE"
MakerDyne 