Been building a site recently which although quite small now will eventually grow to be quite large. When your originally building a site you tend to have a relatively small amount of data. This means you can ‘get away with/accidentally produce’ some less than optimal SQL queries. Its quite difficult to tell where your site will fall down or perform slowly when it has a more realistic amount of data. Or under a more realistic multi user load which by definition as a lone developer its quite hard to simulate.

I tried load testing the site with a relatively old style tool called siege mainly because its quite simple to set up. Sieges makers describe it so

Siege is an http load testing and benchmarking utility. It was designed to let web developers measure their code under duress, to see how it will stand up to load on the internet. Siege supports basic authentication, cookies, HTTP and HTTPS protocols. It lets its user hit a web server with a configurable number of simulated web browsers. Those browsers place the server “under siege”.

You just give it a text file with a list of urls and the POST/GET requests you want made to them, then specify the number of users and how long you want the site sieged for. Then just leave it running at the end of a run it tells you, how many requests it made, how many were unsuccessful etc. Not lots of data but enough to be useful. Mainly I was interested in slow performing SQL so I left my MySQL logging slow queries.

I left siege running for 5 hours whilst I was out and then came back to see how it had gone. Obviously its not absolutely accurate as to what ‘real users’ will do but its a starting point. I found some useful issues that I could then use to target which queries were performing badly. Problems came up where I had never expected. It gave me a chance to improve the performance of some queries before they cropped up in the wild. Also allowed me to check my other queries with a reasonably sizeable set of data.

One thing I did come up though if you do want to set up a relatively long running test like this you probably want to set up any logs your writing on your test server to rotate otherwise like me you will end up with some massive text files that are quite difficult to deal with. But that was OK because I got something large to chuck at splunk …

Obviously you should only do this at your own server/test server otherwise you will just be evil.