Auto-optimizing images in Plone

I’ve been brainstorming a bit about how Plone do an even better job of helping non-technical website editors manage and optimize their images for the web, and I’ve come up with what I think is a pretty clever idea.  I have neither the time nor the skill to implement it, and so I thought I’d float it out there to see if

A) it holds water;

B) if I can’t interest someone in implementing it 😉

The problem: big-ass images

Lots of people who edit Plone websites don’t really understand how to properly optimize images for the web via resizing and compression.   Plone is not entirely unhelpful here; if you upload an image, we automatically create several different resized versions for you to use in your pages.  However, Plone doesn’t currently do anything to help you make sure that original image you’re uploading is appropriately sized and compressed.

When you couple user inexperience with the massive image sizes that today’s digital cameras produce (6/8/10/12 megapixels!), the result is often a situation where users are uploading massive digital camera photos to their Plone sites, then using the PIL resizes in page.  While this works, it quickly produces a bloated ZODB.    In addition, the automatic PIL resizes don’t apply much compression, so the resized images aren’t very well optimized either.  This slows down site performance and causes lots of unnecessary server load.

There are a number of add-on products that can help in various ways, among them:

  • Plone.app.blob and Filesystem storage, which push large binary objects such as images to the filesystem, reducing ZODB bloat.
  • ImageEditor, the amazing Photoshop-like image editing program that runs inside of Plone.  It can do resizing and compression, but only one image at a time.  It’s a great solution for end-users, though.

These are great, but I think we can do better.

Some possible clever solutions

Here’s how I think we can do better.

First, some changes to Plone’s default behavior:

  • Make the amount of compression that PIL applies to its resizes settable in the Plone control panel.  Right now it is hardcoded into ATContentTypes, and it’s hard to customize without rolling your own content types.
  • Automatically resize all “original” uploaded images to 1024px.  Make it user-configurable, and disable-able for those rare occasions when you need to upload really high-resolution originals.

More daringly, I think that someone could write a clever add-on product for site administrators that would do the following:

  1. Walk the site catalog, looking for all objects that are Image-ish and in JPEG format.
  2. Look at the pixel dimensions and the filesize of each image.
  3. Find images where the “bytes per pixel” (filesize/*(height*width)) is higher than a certain “reasonable” value.  (0.5 bytes per pixel seems about right to me, based on the idea that a 150X150 pixel JPEG image shouldn’t weigh much more than 10kb.  (Obviously, these settings should be user-editable.)
  4. Present the user with a list of “suspiciously large” images, along with their pixel dimensions, filesize, bytes per pixel and a preview.
  5. For the JPEG images, offer to apply additional compression to some or all of the selected images.  (Make value user-settable, maybe offer an opportunity to test it against 2-3 images, and estimate the savings you’d realize.)
  6. For images that have larger pixel dimensions than anyone would want to display in a web browser, offer to resize the original image down to a user-specified size (1024 pixels seems like a reasonable default to me).
  7. Apply compression via PIL, presumably in a background process so that Zope doesn’t bog down.  Rebuild Plone’s auto-resized images afterwards from this new original.  Purge caches as necessary.

So, whaddya think?  Is this sane?  Would you get value from these things?  Are there other low-hanging fruit I’ve overlooked?  Interested in helping roll the first two ideas into PLIPs for Plone 4?  Or building out the bulk image optimizer?  Or helping raise some money to sponsor such a project?  I’d love to hear from you.

17 thoughts on “Auto-optimizing images in Plone”

  1. I have actually been thinking about the opposite direction:

    • Move image resizing to WSGI middleware (repoze.bitblt)
    • Original images are stored using BLOBs, so no ZODB DB bloat
    • Get Plone out of the image resizing game

    But of course, we live in slightly different universes. 😉

  2. Sorry about the random formatting above, I assumed this field was plain text — obviously it’s applying some random markup to my beautiful ASCII. 😛

  3. I think that’s a great idea, when Plone eventually moves to a WSGI deployment architecture.

    Much of what I describe above could be done in Plone for Plone 4, then moved out to middleware in a future WSGI-fied release of Plone.

  4. Scaling images on upload is already supported by AT/ATCT, so this would really just entail making that existing configuration (along with the available alternative scales) configurable through the web. Doing this stuff in middleware is probably the best way to go for the future (I’d hate to see much in the way of image manipulation code included in dexterity). For Plone 4.0 using the existing AT facilities would provide all the benefits with much less effort.

    That add-on sounds pretty useful as well. I’d suggest having it use ImageMagick rather than PIL, since ImageMagick offers some very high quality scaling and compression features (comparable to or better than PhotoShop), and PIL is pretty lackluster in that regard.

  5. I like this idea. One possible alternative: A Plone/Flickr (or Picasa, or similar service) integration product. It would add a “Flickr image” or “external image” or maybe just “photo” content type to the “Add new…” menu. (Integrator could disable Plone’s “Image” content type and rename the new content type to “Image.) When you add a new image, it would ask you to select the photo from your hard drive, give it a title, description, and maybe tag it. Then it would work through Flickr’s API (assuming it’s possible) to give the image and metadata to Flickr, and retreive back a URL to store in the ZODB.

    Flickr’s image sizes are pretty reasonable: Square (75 x 75), Thumbnail (100px on longest side), Small (240px), Medium (500px), Large (1024px) and Original.

    Then hopefully the product would allow you to add your Flickr hosted image in Kupu or TinyMCE in the same way as with a “native” image. Bonus if you could upload an image on the fly through the WYSIWYG editor as you can currently do with a native image.

  6. I have this problem on several sites. I have long had it on my To Do lists to automatically scale images to 1024 (or 800) on upload, and to build a tool that would walk the site and perform that conversion after the fact. So I add my support to this idea.

  7. Nice to see some thoughts about images in Plone. I had some things in my research about the resizing of images, so I might have some ideas for you.

    First, I like the idea of helping out the people with the 12megapixel camera’s, who don’t see that there pictures are 5 megs. It’s a benefit for everyone to bring this down. I like your thoughts about setting defaults for images, like auto-resizing to 1024px. If you need bigger pictures than this, it is probably your specialty and going to modify the setting anyway. As a transition solution we can use the info bar to state that the image is uploaded and resized with a link to the settings.

    The idea of making a list of suspiciously large images is great. It will give the webmasters more control over their content.

    Also, I would like to see the image rendering in one system instead of two. Right now you can set the pre-defined sizes for an image when importing in your editor, then set the size again by pulling it sides. This means I can let an image look ugly because I chose 10×10 or render unnecessarily data by choosing the largest size and dragging it small. To end users the dragged size is the only one that matters, why not use this size for rendering or even pick the closest one?

    ImageEditor is great, but it’s for specialized people and we need to work on some good defaults.

  8. I’m pretty sure ATContentTypes does (or did at some point in time) rescale images automatically on upload. OK, it does still mean that you have to process the large tens-of-megabyte images in the first place, but at least it is not storing them.

    On another project I’m working on we’re looking at http://upload.thinfile.com which can split file uploads into chunks and upload them via multiple HTTP POSTs and concat together on the server. I think they have some cunning image-resizing-on-the-client uploader too. Might not be suitable for the general case (esp. licesning) but might be good for specific use cases.

    -Matt

  9. i like all the ideas presented here!
    one more thing from me: i would like to see an easier possibility to customize the image sizes to which original images are converted. currently the best solution is archetypes.schemaxtender, but it’s only accessible for developers.

  10. Hi, As a Plone user rather than developer, I can completely concur with this idea.
    Also, as the images are shrunk down to predefined sizes in Kupu, many non technical users probably believe that as the image is smaller, than the file is smaller. What you see is what you get …. in this case it isnt.

    Simple objectives in usability such as this are incredible important IMHO.

  11. In my opinion using OptiPNG (http://optipng.sourceforge.net/) for scaled PNG images would also help. In many cases it may decrease file size twice, so even less data will be sent to the browser. It is slow, but if the request won’t wait for it, this would be useful for images in a Plone instance.

  12. A couple people have mentioned TTW configurability of image scale sizes. witsch has already implemented this in plone.app.imaging, which I think is a likely candidate for future inclusion in Plone core.

  13. Hi, in one of my content type, I have an thumbnail image I resize to 58×78 on upload, and crop it on center with the following code:

    from archetypes.clippingimage import ClippingImageField
    schema = Schema((
    ClippingImageField(
    name=’thumbnail’,
    widget=ImageWidget(
    label=”Thumbnail”,
    ),
    storage=AttributeStorage(),
    original_size=(58, 78),
    ),
    )
    For the ClippingImageField, see:
    http://pypi.python.org/pypi/archetypes.clippingimage

  14. How about a validation script on the Add Image page which checks for filesize? The user would get a portal message telling them to go compress the file before upload. I’d want to allow a setting in plone_control_panel to adjust what that maximum size should be (100Kb sounds like a reasonable default).

    For hi-res images, my advice is always to upload it as a File content type, so that the site visitor can download the large version instead of having Plone display it on a page.

  15. I may be dense, but why is it a problem for people to upload large images? Normally I want people to upload large images, since that gives me much more room to scale them in various ways when using an image in a rendered page. Since visitors always get server the approriately scaled image the original image size is not relevant for them.

    The only downsides I can think of are upload time, which is a one time penalty that should generally not be a problem, and ZODB size which is easily solved by moving to blobs.

    So my opinions: big-ass images? Yes please!

  16. @wichert-
    In a world where we have blobs out of the box (soon, I hope), the issues with large file uploads will be lessened, but tying up zope threads still sucks. And slow uploads kinda frustrate users (i’d really love to figure out a way to use flash to do the resize on the client, but that feels a bit too ambitious).
    But yeah, poorly optimized images are more of a problem than big uploads.

Comments are closed.