skip navigation

www.Hilands.com


Content:: Setting up Serverstats

Setting up Serverstats
Last Modified: 2013-02-13
This tutorial will show you how to install "Serverstats - rrdtool harvester" on your Linux server. Serverstats is available at http://serverstats.berlios.de/. At this point you should have already installed Apache and PHP, you can use the LAMP installation tutorial to get to this point http://www.hilands.com/os-linux-lamp.html.

Table of Contents
Installing rrdtool
rrdtool is a simple round robin database and graphing utility. It will be handling the core generation of graphs for Serverstats. # apt-get install rrdtool
Configuring iptables
The network graph data in serverstats is generated from the iptables command "iptables -nvxL".
IPTABLES has 3 default "Chains" which are the INPUT, FORWARD, and OUTPUT. We can define what chains serverstats will monitor with the CHAINLIST variable in the traffic.sh shell script which will later be triggered via a cronjob.

When creating the chains we need some type of "dump" chain to get iptables to process the data. We'll name this chain "traffic-output". We will create a chain for each type of data we wish to monitor and tie it into the INPUT and OUTPUT default chains and the traffic-output chain.

The first step is to create a new chain, and flush the chain just incase it already has data in it so we can start off clean. We'll create chains to monitor all traffic, ssh, www, and smb traffic. # iptables -N traffic-output
# iptables -F traffic-output
# iptables -N all-traffic
# iptables -F all-traffic
# iptables -N ssh-traffic
# iptables -F ssh-traffic
# iptables -N www-traffic
# iptables -F www-traffic
# iptables -N smb139-traffic
# iptables -F smb139-traffic
# iptables -N smb445-traffic
# iptables -F smb445-traffic
Now that the basic chains have been created we need to tell iptables what data will go into these chains. Here we'll set up the ports the chains will monitor and tie them into the INPUT, OUTPUT and traffic-ouput chains.
#log all # iptables -A INPUT -j all-traffic
# iptables -A OUTPUT -j all-traffic
# iptables -A all-traffic -j traffic-output
#log ssh # iptables -A INPUT -p tcp --dport 22 -j ssh-traffic
# iptables -A OUTPUT -p tcp --sport 22 -j ssh-traffic
# iptables -A ssh-traffic -j traffic-output
#log http # iptables -A INPUT -p tcp --dport 80 -j www-traffic
# iptables -A OUTPUT -p tcp --sport 80 -j www-traffic
# iptables -A www-traffic -j traffic-output
#log smb 139 # iptables -A INPUT -p tcp --dport 139 -j smb139-traffic
# iptables -A OUTPUT -p tcp --sport 139 -j smb139-traffic
# iptables -A smb139-traffic -j traffic-output
#log smb 445 # iptables -A INPUT -p tcp --dport 445 -j smb445-traffic
# iptables -A OUTPUT -p tcp --sport 445 -j smb445-traffic
# iptables -A smb445-traffic -j traffic-output
These rules should be incorporated into your IPTABLES firewall script. An example script with these rules incorporated can be found at the following link. http://www.hilands.com/code-shell-fw.html
Setting up Serverstats
Extracting
# cd /var/www/
# tar -xjf serverstats-0.8.2.tar.bz2
should now have a folder /var/www/serverstats-0.8.2/
Basic Configurations
# cd /var/www/serverstats-0.8.2/
Copy sample configs # cp -R config.sample/ config
# chmod 777 graph/
# cp sources/traffic/traffic.sh sources/traffic/traffic.sh.orig
# chmod 744 sources/traffic/traffic.sh
# nano sources/traffic/traffic.sh
change the following
LOGPATH="/var/www/serverstats-0.8.2/sources/traffic"
CHAINLIST="all-traffic ssh-traffic www-traffic smb139-traffic smb445-traffic"
SLEEP_BIN="/bin/sleep"
AWK_BIN="/usr/bin/awk"
#${SLEEP_BIN} 50
# cd /var/www/serverstats-0.8.2/config/
# cp simple.php simple.php.orig
We'll now edit the simple.php configuration file. The configuration is a list of php arrays. # nano config/simple.php
On line 72 we have the ping array, 'ping' => array(, to use this we'll change the following settings
'used' => false,
'used' => true,
'hosts' => array('www.google.com'),
'hosts' => array('example.com', 'example2.com'),
On line 80 we have an http ping array 'ping_http' => array(, we can enable it with the following settings
'used' => false,
'used' => true,
'hosts' => array('www.google.com'),
'hosts' => array('example.com', 'example2.com'),
On line 88 we have the traffic array 'traffic' => array(, this is where all of our iptables chains will come into play.
'used' => false,
'used' => true,
this is part of the traffic array group
'chains' => array('WWW'),
'chains' => array('all-traffic', 'ssh-traffic', 'mysql-traffic', 'www-traffic', 'smb139-traffic', 'smb445-traffic'),
'single_bps' => array('used' => false, 'title' => '%s-Traffic (bps)'),
'single_bps' => array('used' => true, 'title' => '%s (bps)'),
'single_count' => array('used' => false, 'title' => '%s-Traffic (count)')
'single_count' => array('used' => true, 'title' => '%s (count)')
Setting up the Cronjobs
We'll edit the crontab and tell it to trigger the update.php file and the traffic.sh files.
Make sure you send the output to dev null "> /dev/null 2>&1" otherwise the php files output will be mailed to you every time it is ran. # nano /etc/crontab
* * * * * root php /var/www/serverstats-0.8.2/update.php > /dev/null 2>&1
*/1 * * * * root /var/www/serverstats-0.8.2/sources/traffic/traffic.sh > /dev/null 2>&1
Fixing errors
You should be able to stop at this point, the following are some of the errors you may encounter and how to debug them.

Deprecated functions in 0.8.2
Version 0.8.2 has some issues with later versions of PHP. 0.8.3 fixes this error. You can manually run the update process with php /var/www/serverstats-0.8.2/update.php
The first error you should see is on the CPU load
Working on cpu
        Updating RRD-file
PHP Deprecated:  Function define_syslog_variables() is deprecated in /var/www/serverstats-0.8.2/includes/logger_syslog.class.php on line 30
# cd /var/www/serverstats-0.8.2/includes/
# cp logger_syslog.class.php logger_syslog.class.php.orig
# nano logger_syslog.class.php
I simply commented line 30 out and everything appears to function as expected.
               define_syslog_variables();
#               define_syslog_variables();
The following errors are about the traffic monitors including the traffic.sh script. I've ran into this a few times during the setup and can rarely find any information about this while scouring the internet. So we'll get into a little bit on how the shell script stores information, how php renders the information, and where rrdtool is configured to store the information.

Starting with some background on shell scripts. There are two ways to "execute" a script in linux, one is if the file is executable it will check to see what the initial part of the script in this case "#!/bin/bash". When the file is executable we can run the file directly. # /var/www/serverstats-0.8.2/sources/traffic/traffic.sh
If it isn't set as an executable file via the file system you will have to trigger the file with the scripting language. # bash /var/www/serverstats-0.8.2/sources/traffic/traffic.sh
It's common for scripts to be triggered via "sh" or "ksh" which is a great way to test the scripts with another shell environment.

There are two errors we can generate one from the php update script and the other from view the broken image file on the website.

If we run the update command php /var/www/serverstats-0.8.2/update.php
the following error will be produced if the data in sources/traffic isn't generated.
Working on traffic_all_traffic
Error:
exception 'Exception' with message 'Could not get current data' in /var/www/serverstats-0.8.2/sources/traffic.php:48
Stack trace:
#0 /var/www/serverstats-0.8.2/update.php(64): traffic->refreshData()
#1 {main}
This message tells us absolutly nothing useful. If we go to the website and attempt to view the image we should see the following error message When we go to the webpage for serverstats and view the graph image directly you may get the following error message if the data isn't being pulled properly.
Error:
exception 'Exception' with message 'rrdtool ("/usr/bin/rrdtool graph '/var/www/serverstats-0.8.2/graph/626e035ae2056bc3d8984bd1650f060e.png' -t 'all-traffic (bps)' -s '-86400' -a 'PNG' -w '500' -h '150' -l '0' -M  -z  'DEF:traffic_all_traffic_bps=/var/www/serverstats-0.8.2/rrd/traffic_all_traffic.rrd:bps:AVERAGE' 'LINE:traffic_all_traffic_bps#7FD180:all-traffic'") finished with exitcode 1
ERROR: opening '/var/www/serverstats-0.8.2/rrd/traffic_all_traffic.rrd': No such file or directory' in /var/www/serverstats-0.8.2/includes/rrdgraph.class.php:420
Stack trace:
#0 /var/www/serverstats-0.8.2/graph.php(248): rrdgraph->save('/var/www/server...')
#1 {main}
The fix for my issue was simple, the traffic.sh script was not being triggered properly due to permissions. If everything is failing try setting the permissions with chmod to read write and execute for all users.
# chmod 777 /var/www/serverstats-0.8.2/sources/traffic/traffic.sh

The traffic.sh script gets the output from iptables with the command "iptables -nxvL" then uses grep, tail, and awk to read the amount of data that has been sent to a specific chain. After the data has been found it is then stored in a "log" file in sources/traffic. The name of the file will be the name of the chain we created with iptables.
When the php update command is ran php /var/www/serverstats-0.8.2/update.php
Serverstats will then generate the rrd records stored in the rrd/ folder. The name of the rrd database will be named "traffic_" followed by the chain name with the extension .rrd. The error we received on line 64 appears to be a class that triggers a function named refreshData "$source->refreshData();". The processing for this class is done in the simpleconfig.class.php file on line 827.

The final processing for the png image is generated when the web page calls the graph.php file. Caches of the images are stored in the graph/ folder, if you recall above we changed the permissions for the graph folder to 777. This will allow the webserver user to write data to this folder, it would be recommended to set the folder to the user or group of the web server and modify the permissions to read only (4) for everyone.

Graph Errors in PNG's
If you can't see the PNG files and open them up and get the following error message
ERROR: Could not save png to '/var/www/serverstats-0.8.2/graph/7774f52e92310efce731acbfe4d6ab5e.png'
581x237' in /var/www/serverstats-0.8.2/includes/rrdgraph.class.php:420
Stack trace:
#0 /var/www/serverstats-0.8.2/graph.php(248): rrdgraph->save('/var/www/server...')
#1 {main}
Your permissions are not allowing the graphs to be saved. You can use chmod and set the permissions to 777 as a quick fix. I'd recommend setting the user and group permissions with # chown -R www-data:www-data /var/www/serverstats-0.8.2/graph then setting the mode permissions to 664 # chmod -R 774 /var/www/serverstats-0.8.2/graph Alternatively you can touch the files that aren't showing up and chmod them to 664 and set the correct owner/group.
Syslog error
Feb 12 18:26:01 crossroads /USR/SBIN/CRON[22682]: (root) CMD (php /var/www/serverstats-0.8.3/update.php > /dev/null 2>&1)
Feb 12 18:26:01 crossroads /USR/SBIN/CRON[22680]: (CRON) error (grandchild #22682 failed with exit status 255)
# php /var/www/serverstats-0.8.3/update.php
Working on cpu
        Updating RRD-file
Working on load
        Updating RRD-file
Working on memory
        Updating RRD-file
Working on postgresql
Working on cpu
        Updating RRD-file
Working on load
        Updating RRD-file
Working on memory
        Updating RRD-file
Working on postgresql
At a first glance it looks fine no errors. Looking again it seems to hang on Working on postgresql.
# nano /var/www/serverstats-0.8.3/configs/simple.php
Search for postgres and comment out the section with /* */
/*
        'postgresql' => array(
                'used' => true,
                'host' => 'localhost',
                'port' => '5432',
                'user' => 'postgres',
                'password' => '',
                'dbname' => 'template1',
                'graphs' => array(
                        'transactions' => array('used' => true, 'title' => 'PostgreSQL: transactions per second'),
                        'backends' => array('used' => true, 'title' => 'PostgreSQL: backend count'),
                        'tuplesread' => array('used' => true, 'title' => 'PosgreSQL: tuples read'),
                        'tupleswritten' => array('used' => true, 'title' => 'PosgreSQL: tuples written')
                )
        ),
*/