Some not terribly complete thoughts on some ideas for how the Plone community could make it easier for mainstream hosting VPS companies to offer Plone to their customers.
The problem: Plone is relatively difficult to host in the most common and inexpensive commodity web hosting accounts.
Why? Plone, like Ruby on Rails and other Python frameworks, requires long-running server processes. These processes consume relatively large amounts of RAM (256MB or more), and thus don’t “play nicely” in a typical chroot-based virtual hosting environment optimized for PHP and Perl, where hundreds of accounts are crammed onto a single server.
The elements of a better solution are lying around on the table, yet they have not been assembled. This article attempts to point out some possible ideas for innovation.
Virtual private servers (VPS) give a user true root privileges on their own instance of the operating system. Typically, the main limiting resource on a VPS instance is the amount of committed RAM. 512 MB of RAM is a good amount for a small to medium-sized Plone site under not-too-large loads.
Xarope is a set of scripts by John Lenton and Marcos Dione of Except, which automate the creation of Xen VPS instances with pre-configured Zope instances.
Also worth taking note of is the fact that Cpanel is a very popular web-based GUI for server management amongst commodity hosting providers. Its companion, Fantastico, make it possible to have “one-click” installation of many popular PHP and Perl applications.
My hypothesis: it should be possible to create a virtual machine (Xen) image that contains the following preconfigured elements:
- CentOS or other commonly used linux operating system suitable for virtual machine operation
- Python 2.4.x (or other current version suitable for Plone)
- Other python packages required by Plone (PIL, OpenID, elementree, setuptools, etc.)
- Apache or NGNIX, with rewrite rules pre-configured or script-configured for a basic single-site deployment
- Varnish or Squid, appropriately pre-configured or script-configured for a single-site deployment.
- Scripts to configure Apache/NGNIX/Varnish/Squid for your domain.
- The main question the interactive auto-configuration script should ask is: “What is your domain name?” it should assume you are hosting a single Plone site on the server, at the domain name specified, IOW sensible defaults, further customization assumes you know how to configure Apache, etc.
- It might be ideal to have these scripts be triggered via the CPanel or Fantastico web GUI, or some equivalent.
- Plone 3.x, with sensible configuration defaults for this environment, e.g CacheFu.
- The service provider might also choose to bundle some popular add-on products such as PloneFormGen, LinguaPlone, etc.
- Appropriate utilities for automated backup
With this image in hand, we should approach well-regarded VPS providers, and work with them to make this “Plone Image” available as an option for their customers. I’d like to see a target price of $30-50/month for 256-512 MB of RAM.
I’d also like to see the instance have CPanel for basic server administration (e.g. email, apache settings, other non-Plone stuff), whatever the VPS provider usually providers for its customers.
Sidebar: if Repoze + WSGI provides an effective way to run Zope in an Apache mod_wsgi process, this could be an alternative configuration. This doesn’t get us around the need for a generous amount of RAM, but it may be more understandable/palatable to hosting providers, and is worth exploring.
I’m not sure how mod_wsgi would do in terms of memory requirements. More particularly, I’m not sure if you can run mod_wsgi with a threaded Apache server. And if you are using worker processes then the memory use is going to be really big, as you have multiple entirely separate copies of Plone loaded into memory (using a shared ZEO instance). Also, I’m the process management for ZEO remains something of an issue (though ZEO is usually very well behaved, so it’s not really any worse than MySQL). Also the way mod_wsgi usually is setup, it brings up new processes and takes down old processes periodically. This is fine — good even — as long as your server can come up reasonable quickly (e.g., in a second). Plone isn’t like that.
Good Plone hosting is also available with a few existing shared hosting companies. WebFaction and Zettai are two of them.
Although Xarope was my idea way back then, the person who actually wrote the scripts was Marcos Dione.
Would webmin do instead of cpanel? Or any other Free software, actually.
Hi Jon, good writeup. One minor modification at the end: the goal will be Repoze+mod_wsgi, not mod_python. Graham, the mod_wsgi guy, has been doing a lot of writing and thinking about Python and web hosting and has lots of material on the subject.
@Ian: thanks for the clarification; I guess the bottom line is that we will need to do a lot more to make Plone and Zope less memory-thirsty if we want squeeze it into ultra-low-end hosting (not that I personally think that is the be-all, end-all.) Support from big VPS hosters would be fine with me.
@Mike: Webfaction, HSR and a few of the other “Plone specialist” shops are indeed good folks; the question I’m interested in is how to radically expand the choice in the market.
@John, thanks for the correction!
@Paul, thanks for the correction!
I’m doing some work toward a departmental hosting service [https://weblion.psu.edu/trac/weblion/wiki/WebLionHosting] at Penn State University. While the image-based approach gets you the initial installation, what will you do about upgrades? That’s where most of the challenge lies, IMO. We plan to run Debian on a bunch of VMWare VMs—one per department, since our departments operate almost autonomously—and manage both installation and upgrades using the native package management tools. We’re starting with a base of the excellent Plone 3 packages by Fabio Tranchitella and the rest of the Debian Zope team and adding packages of our own to handle things like Apache config, additional Plone products, and our weird single-sign-on system. With careful planning, I think we can get minor Plone and product upgrades happening unattended.
@Erik — great point! I’m really inexperienced with VMs, but I agree, how to maintain and upgrade packages is a key question. Buildout may have some role to play. I’m not sure if it is easy to re-save the VMs as updated images. I would imagine the current VPS providers have some means for doing these kinds of tasks. I’d love hear what others know.
For the last three months, I have been slowly migrating my plone sites to VPS at http://www.slicehost.com . Coming from a high end managed hosting environment, I find it refreshing to be able to create custom servers tailored to their job. (I could not afford this with managed hosting, where I had a server that did everything.)
The Slicehost guys commented in a podcast, that they have built the infrastructure to deploy VPS images pre-built (called stacks) for a purpose. They had intended to provide Rails images, because Rails was difficult to install. They have decided not to offer such images for the time being. As I recall, they feel that in the last year Rails has become both much easier to install, and has many more install options. So the need for a “standard” Rails image has been reduced, at the same time it is less clear what the “standard” would be.
If standard, community backed server configuration could be defined, I think that we could approach that Slicehost and other VPS’s to provide base images.
However, I wonder if we wouldn’t be better off extending the installer script from the Plone Unified installer to cover much of the base server setup. As I have been creating VPS systems, I have been documenting the process with the hopes of building a shell script to automatically build the entire server environment from a fresh Debian vps.
Hi Jon,
Unfortunately, software doesn’t address the biggest issue in plone hosting – RAM. CSquared provides clients with Xen based VM’s for plone – and at a minimum you need 512M RAM.
Image creation is easily automated. Updates can be handled by COW (copy on write) file systems, or automated scripts. Administration can be handled by plone_control_panel configlets
But there is no getting around the RAM. And it needs to be physical RAM – swapping just kills Plone.
Even worse, as Zope/Plone is a long running process (as you mentioned…) it eats that RAM even when its not being used. as opposed to (f)cgi based systems that only need the RAM while the script is actually running….
@mech422: since VPS accounts with 512mb of RAM are now priced around $40/month, I don’t consider the cost of RAM to be much of barrier anymore. It’s all about making it really easy for a novice integrator to get a Plone site up and running fast.
Slicehost is a great option and they now office slice cloning, so you can take a copy of a slice, clone it and set up another one. This would be great if you could set up the basic slice for Plone (os/tools/installs/buildout) and just clone it to bring up another Plone slice.
I would like to see some great documentation on setting up multiple slices for Apache/Varnish/Plone/ZEO/Zope/etc to handle multiple Plone sites out of one Zope instance.
Apache -> Varnish -> ZEO -> Zope1/2/3 – PloneSite1/PloneSite2/PloneSite3
BZ
I love this thread…It would be incredible to have Plone appear as a choice on cpanel at hosting companies – or even just a viable choice for hosting companies. It’s really too bad that the memory requirement is so steep, though. From a hosting company’s point of view, the other red flag is that all it takes is one code problem and the running python instance can just eat all the memory up in a heartbeat. So, supporting “Products” as add ons would require testing and verification. But supporting users with this weakness is also a potential source of unwanted support calls. And you also need ssh access, of course, and that’s still often an extra cost (but not much) so it’s not a “base package” situation in all cases to have plone as a hosting company option.
As much as I like Plone, modern methods of traffic generation and feeder site infrastructure just do not lend themselves to using Plone, so the target market is not going to be internet marketers who just want to slam up a site for authority generation and be done with it.
So, that just begs the question — who is zope/plone really targeted at and what kind of applications/solutions is it intended to solve? This would help focus the discussion on commercial plone hosting solutions.
Never been tried Plone in my debian vps. I will try it today, thanks for this wonderful article. Now, I am ready to use it on my 3GB RAM.
Also, “Debian VPS” is a good name for a techno band.
I’m not sure if you can run mod_wsgi with a threaded Apache server. So, supporting “Products” as add ons would require testing and verification. But supporting users with this weakness is also a potential source of unwanted support calls.