Monitor website data usage

Hi

Does anyone know of any good software to use to track and monitor data usage/bandwith for websites ?

We have about 50 sites offering unlimited data but want to curb this and allow say 3GB/per month per site.  

Any recommendations of production grade software that will do this ?

Thanks

CobraitiAsked:
Who is Participating?
 
xtermConnect With a Mentor Commented:
First get and compile and install mod_watch into your Apache, most likely /usr/lib/httpd/modules/mod_watch.so.  You can download the source here:

ftp://ftp.grandecom.com/incoming/mod_watch-4.3_apache22_mod.tar.gz

Set apache to load and configure this module by doing the following in httpd.conf:

LoadModule watch_module       /usr/lib/httpd/modules/mod_watch.so

<IfModule mod_watch.c>
    # Allows the URL used to query virtual host data:
    #
    #   http://www.snert.com/watch-info
    #
    <Location /watch-info>
    SetHandler watch-info
    </Location>

    # Allows the URL used to query file owner and web
    # server data:
    #
    #   http://www.snert.com/~achowe/watch-info
    #   http://www.snert.com/~SERVER/watch-info
    #
    <Location /~*/watch-info>
    SetHandler watch-info
    </Location>
</IfModule>

After that, you should be able to grab mod_watch.pl from here:

http://www.sigterm.net/projects/old/FreeBSD-5.x/mrtg/mod_watch.pl

This basically does a grab of the statistics for a given site on your server, and outputs them in MRTG style format if you want to use MRTG to graph them.  However, you can either hack it up to pump it into a SQL database, or overlay a shell script to get the values to do whatever else you'd like with.

Start with that, and let me know if you have questions.  The one thing I'll tell you that I ran into is that I got some weird Apache errors that looked like this:

[Thu Oct 21 11:56:48 2010] shVerifyString(2af44d641600, 2af444cb36f8) failed!
[Thu Oct 21 11:56:48 2010] [crit] (20014)Internal error: shGetLockedEntry(2af44d641600, "isys3.lsn.net") failed in watchLogHash()

If that happens to you, you will have to rebuild mod_watch but patch a single line in the code first according to this page:

http://www.ovcharov.me/2009/09/01/kak-pochinit-mod_watch/

It might sound like a lot, but it was really easy, and I'll help you if you get stuck.  Now I have this sweet web page that shows all my virtual hosts, and how much transfer they've done every month for like the last 5 years :)

0
 
PapertripConnect With a Mentor Commented:
Awstats is a great tool for this, been around a long time.
0
 
CobraitiAuthor Commented:
Thank you very much for that

Do you know of any paid and supported software used by web hosting companies as well ?

Thanks again
0
Get expert help—faster!

Need expert help—fast? Use the Help Bell for personalized assistance getting answers to your important questions.

 
xtermCommented:
I recommend webalizer for log analysis for generating graphs and whatnot to provide to your own site owners.

However, if you want to limit bandwidth, you should set up mod_watch to collect (and perhaps populate a database of) statistics.  This way you can set up something to alert you if somebody has exceeded their monthly transfer limits.  It is not difficult to implement if you'd like some further direction in that regard.
0
 
CobraitiAuthor Commented:
Hi xterm,

That would be awesome if you could point me to some related articles.  I really need to monitor website data transfer and alert users once they've reached their bandwidth quota.  

Thanks again.
0
 
PapertripCommented:
Sorry I completely overlooked the b/w throttling part of your question ;)
0
 
CobraitiAuthor Commented:
Thanks guys for all your suggestions.  Very much appreciated :)

I
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.