storing apache access logs in a mysql database on debian October 27, 2007
Posted by chris in : SQL, linux , add a commentMod log sql is an awesome way of getting away from those old log files and is really handy for both web development and system administration. It’s been a while since I’ve posted a blog and this is something I’ve never done before so here we go. If you are ever doing any kind of parsing of your apache access log and run a relatively high traffic website (the one I’m doing this for, mp3crib.com, averages over 5,000 hits a day and some days gets over 10,000) you will begin eating up huge amounts of CPU and memory (if storing the information in an array). Well lately it’s gotten so bad that my PHP script fails due to memory exhaustion. I can’t have that. I heard somewhere that databases are better than flat files, go figure. If I knew a lower level language then I would of course write my parser in that…but I don’t. So luckily libapache2-mod-log-sql exists.
(more…)