Upgrading the Software Behind RIPE Labs

Since RIPE Labs 2.0 , we have used the open source content management system (CMS) Plone as the technology to power RIPE Labs and we've been very happy with the software. Recently, the Plone core development team released a new version of the CMS and we decided to implement for the following reasons:

• Improved editing experience
• Enhanced performance
• Security updates 
• Newer software stack for the RIPE NCC to build features on

We want to share our experiences of this upgrade and inform you of some editorial changes. We also plan to give feedback to the Plone community about the issues we experienced during the upgrade.

What Has Changed?  

Visitors to the site will experience no difference at first. If you choose to contribute a RIPE Labs article, however, you will see a new editor interface. 

Gone is the green-coloured editor area. It has been replaced with a more modern side editor bar which will help you as a user to visualise the content with an uninterrupted view. The editing bar can even be minimised if required.

Old

New

All of the previous options are now available on the left-hand panel.

If you would like to create new content, you can still do this via the "Add new" link.

To find additional information on contributing to RIPE Labs, please read our how to page.

The upgraded visual editor (based on  TinyMCE ) is a lot easier to use and has a cleaner, crisp look. There is now an option for bulk uploading of files and images, this includes a drop and drag option as shown below. This is a feature which has been requested by several of our users and can be found within your member folder contents area.

It should be faster to upload content now and we will keep looking into ways of making this even better. 

Please let us know if you find any bugs or experience any issues with the upgrade by using the "Report a bug" button at the bottom right corner of this page.

The Plone 5 Upgrade Experience

While preparing for this upgrade, we realised that our team was one of the first to upgrade a large Plone installation to the latest version and as such, we ran into issues that were not well documented. We also found features that behaved in unexpected ways. Below, we will go into technical detail on our experience upgrading from Plone 4 to Plone 5 on a couple of large, heavily customised websites.

Implement a Two-Stage Migration

As Phillip Bauer mentions in this video , it’s best to migrate to the latest point release of Plone 4 and then to Plone 5. While doing so, you should remove plone.app.iterate from Plone 4 and install Diazo if you haven’t already. This helps in the long run.

 could not upgrade calendar_future_years_available property
 

 ...
 

 KeyError: 'Interface `plone.app.iterate.interfaces.IIterateSettings` defines a field `checkout_workflow_policy`, for which there is no record.'

As some of our sites still use Archetypes, we did a ZEXP export of the reference_catalog  before wiping it as any entries in the reference_catalog will break the Plone 5 upgrade steps. The reference_catalog.zexp can be imported later once the Plone 5 site is up and running.  

RequireJS and Plone Legacy Resources

As it's been well documented, the Plone core implementation of external resources doesn’t really work with additional add-on products:

• https://community.plone.org/t/again-dealing-with-3rd-party-js-modules-in-plone-5-is-a-major-pain/1269

We also ran into several issues regarding registering our own JavaScript libraries:

• https://community.plone.org/t/issues-with-upgrading-to-use-registry-xml/2237
• https://community.plone.org/t/plone-5-upgrade-failing-on-resource-plone-css/2074

The need for an Asynchronous Module Definition (AMD) solution within a modern CMS is understandable. However, a vast amount of our development time was spent trying to set this up with our existing custom code while avoiding clashes.

You might see the common define() mismatch error. The only way we found to solve this was to reference  http://requirejs.org/docs/errors.html in order to work out the clashes and go through the stack to see what custom resource is being loaded too soon:

Our recommendation is to remove any legacy Plone resource files which have been created via the upgrade. For us, these were just getting in the way, especially if there were any Plone 4 CSS or JavaScript files with conditions. An example of this is if you wanted to show only a given resource for authenticated users. Then this condition would be dropped during the upgrade and you would often have resource clashes.

To fix the legacy bundle issue, we ran an upgrade step to purge our portal_javascript and portal_css registries. Then we added the following code to the registry.xml located in our site's policy package to make other non RIPE NCC owned files implement correctly:

 <records prefix="plone.bundles/plone-legacy"
 

 interface='Products.CMFPlone.interfaces.IBundleRegistry' purge="True">
 

 <value key="merge_with">default</value>
 

 <value key="resources" purge="false">
 

 <element>jquery-highlightsearchterms</element>
 

 </value>
 

 <value key="depends">plone</value>
 

 <value key="jscompilation">++plone++static/plone-legacy-compiled.js</value>
 

 <value key="csscompilation">++plone++static/plone-legacy-compiled.css</value>
 

 <value key="last_compilation">2014-08-14 00:00:00</value>
 

 <value key="compile">False</value>
 

 <value key="enabled">False</value>
 

 </records>

This was done while the site was using the Plone 4.3.10 version.

RequireJS is a useful tool once you get used to it, but you have to watch out for the your custom libraries load points compared to the default Plone tools. A simple $.document.ready() is not enough anymore.

Archetypes and Site Properties

As previously mentioned, we did the initial upgrade with Archetypes in place. The lock_on_ttw_edit and ellipsis options were missing from site_properties .

These were added back to the site_properties registry while we wait to upgrade to Dexterity.

Sample errors:

 AttributeError: lock_on_ttw_edit
 

 - Expression: "lifecycle/begin_edit"
 

 - Filename: ... 7.egg/Products/Archetypes/skins/archetypes/base_edit.cpt
 

 - Location: (line 6: col 21)
 

 - Source: dummy lifecycle/begin_edit;

and:

 NotFound: ellipsis
 - Expression: "site_properties/ellipsis"
 - Filename:   /ripe/default_error_message
 - Location:   (line 69: col 22)
 - Arguments:  repeat: {...} (0)

Unsupported Add-Ons/Customisation Is a Pain

It’s the nature of the beast to have custom elements as part of any open source CMS. We try to keep them to a minimum but bespoke code is dotted throughout our sites.

That said, it can a bit daunting to upgrade the site to the latest version of Plone and notice that you have more than 400 tests to fix.

Some of the add-ons which we use were built in-house, but some have been installed since 2011 and working correctly until the recent upgrade. Where possible, we forked and submitted a Plone 5 patch (for example https://github.com/AdCastle/qi.portlet.TagClouds  ), but we were surprised at the number of unsupported products for Plone 5, including the commonly used packages collective.indexing and collective.solr !

Custom Portlets

While upgrading, we experienced portlet complications.  We couldn’t find anything documented about this issue, so it could just be our customisations. But due to a change in zope.formlib, we were receiving the following error:

 NotImplementedError: The class deriving from AutoExtensibleForm must have a 'schema' property

A sample portlet code which would cause this would look something like:

 class AddForm(base.AddForm):
    """"""
   form_fields = form.Fields(IInterfaceExample)

The change is pretty easy as the schema property must now be set for editable portlets.

 class AddForm(base.AddForm):
""""""
    schema = IInterfaceExample
...
class EditForm(base.EditForm):
    """ """
    schema = IInterfaceExample
...

If you would like to use any of the new pattern driven widgets in your portlets, you will need to use the CatalogSource library for any of the multiple choice widgets (i.e. if you have used schema.Choice or schema.List ).

The code changes from:

 from plone.app.vocabularies.catalog import SearchableTextSourceBinder

to:

 from plone.app.vocabularies.catalog import CatalogSource

Then the custom portlet query needs to be redone using the CatalogSource method.

Good Documentation Helped the Upgrade Experience

There was a whole host of documentation provided by the Plone community, which helped with the migration and it was a good starting point compared to the Plone 2 to Plone 3 migration from years ago.

There is also a very good presentation by Philip Bauer on this topic given at the Plone conference in Bucharest. We think it's important to share experiences (good and bad) within an open source community which is why we tried to document them within this article.

Plone 5 Upgrade Completed

Though the upgrade has taken some time, we hope you enjoy the improved website. Plone 5 has some excellent new features and we are certainly appreciative to the Plone community for their work. We hope our findings shared here will help others in the Plone community with their upgrade and development work.