Server-Friendly WordPress: A Year With Squid

Entropic Memes is by no means a high-traffic site, even for a personal blog. However, it’s occasionally high-enough traffic that performance issues can occur, were steps not taken to prevent them. For this website, those efforts comprise four distinct efforts: Application-level output caching, server-side caching, scalable architecture, and a transparent caching proxy front-end. Each is designed to minimize the impact of one or more specific performance bottlenecks that have arisen over the years.

Today, having just hit the eight-hundredth post on this site, and survived the highest sustained level of traffic the site has ever weathered, I thought it’d be a good time to look – once again – at the last of these measures, the transparent Squid proxy that serves up all the requests for this website, and which has been in use for just over a year.

I first implemented Squid on this site after reading about it’s dramatic effect on “HD-DVD night”. Like the guy at that site, I’d already enabled WP-Cache, and made other optimizations to keep things running smoothly on this site. Like him, I also ran into problems where Apache just couldn’t keep up with the incoming requests. Unlike that site, Entropic Memes shares its server with a number of other sites, so has to play nice, and hogging all the available Apache processes is very not cool.

Thus, Squid.

So what’s it do? Well, it sits between the webserver (Apache) and the world, and “handles” incoming requests for “objects” – files of whatever sort, be they text, images, or something else – from web browsers. Those which are frequently requested get cached into memory, and “served” by Squid – without the bloated monstrosity that is Apache ever having to know about it; requests that aren’t cached get passed transparently to Apache.

On this site, simply using Squid means that the number of requests handled by Apache dropped by more than ninety percent! For most pages that are requested, nineteen “objects” are requested – the page, a stylesheet or two, and a bunch of graphics. All the graphics that aren’t part of the content of a post are served from the cache – from RAM – better than 99% of the time. (Images in posts are served from a separate server, elsewhere.) The actual HTML content of a page is cached for a few minutes at a time; this can be a little wasteful, when someone visits one of the eight-hundred (and counting!) pages from the site’s archives. On the other hand, when one post gets really popular – as happened this week with this post – it means that literally thousands of requests can be served – effortlessly, and very quickly – by Squid, without Apache ever seeing a thing.

Care for some numbers? I ran a log analyzer on the Squid logs for this past Sunday, Monday, and Tuesday – the latter two days were among the busiest in this site’s history – and got some numbers:

As you can see, Squid served, over three days, 105,318 requests from a total of 6170 hosts. Of that, 91,518 requests were served by the Squid cache, from memory, without ever touching Apache. Nice, eh?

For those who are big into Squid stats, here’s the breakdown by request type: “Hits” are objects served direct from the Squid cache, while “misses” are objects that had to be served up by Apache.

And the breakdown by file type:

That’s pretty self-explanatory, really.

So, after a year with Squid, I’m – understandably, I hope – quite pleased with the setup. Yes, there are other ways to achieve similar – and even better – performance, but I’m content with what Squid can do. If Apache – or the memory Apache uses – are bottlenecks for your site, consider giving Squid a try, if you can. The results might just surprise you. ๐Ÿ™‚

Published in: Geekiness, Meta | on June 4th, 2008| 2 Comments »

Both comments and pings are currently closed.

2 Comments

  1. On 4/25/2012 at 11:16 pm Tom Said:

    This all sounds great. Do you know of any good tutorials on how to set up Squid for a WordPress site?
    I’m looking for ways to optimise my sites and like the sound of squid. I’m just finding it a little difficult to find some simple instructions…

  2. On 4/24/2013 at 10:46 pm casino online Said:

    casino online…

    ยป Server-Friendly WordPress: A Year With Squid – Entropic Memes…