Plone 4 is about to leave its alpha testing phase and enter beta testing prior to a final release. Update: Plone 4 has made a final release; I’ve updated the graph.
One of the many things the Plone team has worked really hard on in this release cycle is improving Plone’s performance. Plone core developer Hanno Schlichting has blogged about this a number of times, and deserves a tremendous amount of individual credit for digging deep into Plone’s innards to find and fix inefficiencies.
When I read Hanno’s most recent performance post, in which he shows Plone 4.0 alpha 3 serving up 22 pages/second without caching on his personal laptop, I started to wonder how this compares with some of the other common CMS platforms out there. I was pretty sure none of them could deliver more than 10 pages/second with zero caching or performance tuning. So I did a bit of quick-and-dirty speed benchmarking.
I’ll start with results, then explain my methods. As I suspected, Plone 4 is faster out of the box than some of the most common PHP platforms. Lots faster.
That’s right: Out of the box, Plone 4.0 served up 15.1 pages/second, that’s over three times faster than Drupal 7 alpha 1′s 4.1 pages/sec, Joomla! 1.5.15′s 3.6 pages/sec or WordPress 2.9.1′s 4.5 pages/sec. Plone 3.3.6 our currently shipping release, turned in a snappy 9.4 pages/sec, over twice as fast as its PHP competitors. Whee!
Here’s how I tested. Hardware was my 2GHz MacBook with 2GB RAM. I installed Plone 4.0 using buildout, and used MAMP 1.7.2 to run the PHP products. I did a default install of each product, no add-on modules, I used whatever default content and initial configuration each product provided. (Joomla! gives you the option whether or not to install sample content, which I accepted.) No caching was installed or configured for any system. I measured performance of the homepage as an anonymous user with trusty ol’ ab -n 10, ran it a few times to get the systems warmed up, then noted the value where the runs stabilized.
Lies, damn lies and statistics
So, what does all this mean? Well, honestly, not much. (Although Hanno clearly has a faster laptop than I do!) This is obviously a crude benchmark. I didn’t load up each CMS with realistic sample content. I tested two pre-release products (Plone 4.0 alpha 3 and Drupal 7 alpha 1) against three production releases (Plone 3.3, Joomla! and WordPress). All of these CMSes can easily pump out hundreds of pages per second with a little bit of tuning, reverse proxy and/or database caching (although many real-world users don’t bother with performance optimization!). So, this benchmark is definitely not an accurate measure of the real-world performance of a site. (Matt Hamilton recently explored this a bit.)