Adapting to ever changing user needs is difficult with a traditional CMS. A modular approach to software reuse helped Red Badger deliver a great experience
Adapting to ever changing user needs is difficult with a traditional CMS. A modular approach to software reuse helped Red Badger deliver a great experience to both Sky’s customers and their editorial staff.
Building a content-managed website is more challenging than ever. With the agile, iterative approach to product development, it is the customer behaviour that is now driving the way a site evolves. It is therefore vital to have the flexibility to change any and all aspects of the user experience quickly, while allowing editors to easily create and change the content itself in a user-friendly way.
Traditionally, organisations approach the challenge by taking an off-the-shelf content management system (like Drupal, Wordpress, OpenCMS, etc.) and building their website using it. The existing CMS products define a pattern of content storage and presentation that can be changed, but only in a limited number of ways because it is fundamentally constrained by the core design of the system.
It was these fundamental constraints that made us decide to build the Help & Support website for Sky and its corresponding CMS from scratch and take a very different approach to building a content management system. And even though we decided to build a bespoke CMS, we still aimed for it to be reusable across Sky (and potentially beyond).
Buy or build?
Any time you set out to solve a problem that others have already attempted to resolve, do you try and start with what has already been done or do you try and take a new approach and start from scratch?
The first option means that everything the existing solution doesn’t do out of the box you will need to do with plugins, tweaks or outright hacks. In effect, you are shoehorning your requirements into it. That of course doesn’t just hinder development speed, it also means issues around long term maintainability and overall cost.
The custom build option has obvious benefits in flexibility. You can build exactly what you need and it is very easy to change later because you have full control and ownership. On the other hand, if an existing solution covers a large portion of the requirements, you may be reinventing the wheel and needlessly rebuilding existing functionality.
You could argue that the right solution is to pick the tool which does what you need up front. The problem with that approach is that it really isn’t your needs that matter but rather the needs of your customers and not only do you usually not know what those needs are up front, but they have an annoying habit of changing constantly. What fits in the beginning, therefore, may not fit six or twelve months down the road.
At Red Badger we started off by first gathering all of the requirements we knew of at the time and placing them along two axes: from off the shelf to bespoke and from nice-to-have to must-have. Most of the hard problems (e. g. revision history or publishing workflow) ended up in the bespoke must-have corner. This gave an objective view that bespoke could provide both greater value and speed to market.
Having made the decision to go for a custom build, we then had to come back to the issue of reusability. A website with a CMS is one of the most common projects our customers ask for, so reusability is high on the list of priorities. Plus for the solution to be really valuable to Sky they would want something that could be reused elsewhere in the business.
We didn’t want to attempt to create a CMS that could be all things for all men either, because we believe it fundamentally cannot be done. The problem with the CMS is in the definition of the term, or rather the lack thereof. A universal CMS is like a universal remote control: at first it looks like all remote controls are the same - a small plastic box with some buttons that control various functions. It should be easy enough to have just one that caters to both or even all similar problems.
On a closer inspection though, you realise a volume control doesn’t make sense for a DVD player and an eject button doesn’t make sense for a TV. A TV remote fits perfectly to the requirements for controlling a TV, a DVD remote fits perfectly to the requirements for controlling a DVD player. Put them together and you only end up with a situation even more confusing than the two remotes you started with.
In the same way, trying to create a universal CMS is a fundamentally flawed goal. The total cost of ownership will be lower with a less universal and more specific solution, especially if you build on an open source foundation. This is a general issue with enterprise software you are meant to buy, deploy and use. It never quite works and it all comes down to software reusability in general.
Software reusability
The first instinct of any software developer is to build everything from scratch. It is easy, can be started right away, everything is simple and understandable and crucially, no choices have been made yet which means everything is open. However, because there are many risks involved (software projects are famously hard to keep on track in time and budget when not properly managed and executed) the first instinct of a business is to look for an existing, off-the-shelf solution. The cost is known, it is available right away and after all there is no reason to reinvent the wheel.
The difficulty is that this thinking only applies to well defined and simple problems (like a wheel, for instance). The creation of software consists of a long series of decisions based mostly on assumptions and best guesses. When those choices are made, everything else is based on them and can’t be easily changed later on.
Furthermore, software has to be designed for reuse. That means defining the problem as specifically as possible, considering all the options and use-cases and inventing an abstraction that covers all of them. The bigger and more varied the problem, the harder it is to find a good abstraction.
The biggest issue with traditional off-the-shelf content management systems is they attempt to solve a problem that is an umbrella term for a large set of related problems, each with several distinct parts. The best solution to each of them is case-specific, i.e. in each particular CMS the best solution will be different.
So, it sounds like our quest for reusability is doomed. Fortunately, there is another option. We can take the approach software developers (and all engineers, really) take below the surface and bring it all the way up to the top. Rather than aiming to reuse the full product, we can build a set of reusable tools that can be combined in different ways to quickly form a new solution to a new, similar (but not the same) problem.
No software system is actually built from scratch any more. They are far too complex and large. Even custom solutions are built from about 90% reusable tools and 10% custom code that utilises those tools to deliver the user facing features. And because the tools are small and specific, they are much easier to design for reuse.
We built the Sky Help site and CMS in exactly that way - from small reusable components responsible for simple tasks that can be reassembled to create bespoke content management systems quicker without losing the flexibility.
At the end of the project we even released two of those components as open source software: a CMS kernel / content storage backend called Colonel and a WYSIWYG editor called Faber.
So what is a CMS, anyway?
Broadly speaking, a content management system has three main functions: storing the content, editing the content and publishing the content. Each of them are distinct with their own set of challenges, yet most traditional content management systems attempt to solve all three at the same time.
Building a coherent whole that aligns the three functions in a sensible way means making many decisions resulting in a highly opinionated whole. This results in tight coupling which makes the CMS hard to setup, customise and deploy. Vendors then usually attempt to bring back some of the flexibility by adding a multitude of complex and interrelated configuration options and plugins of varying quality.
Instead, Colonel is designed to make as few assumptions as possible while handling the basic functions of a modern CMS. Combined with other small and focused tools (often also designed for reuse), it can deliver a solution which fits the specific requirements for a CMS being built at the time. However, this means that on its own, Colonel doesn’t provide a complete system. That is, instead of building the specific CMS and website in Colonel, you build it around (or using) Colonel.
This way of building a CMS by combining different tools encourages creation of multiple micro content management systems each focused on a small number of editing tasks and publishing workflows. The quality of user experience is not compromised and enhancements are easier to make as the complexity of each micro CMS is low in comparison. Finally, the individual systems can cooperate and share content over a well defined web API, further promoting sharing and reuse (e.g. a shared image CMS for the entire organisation).
Let’s look at what the basic functions of a CMS are to better understand what reusable tools we can design and how Colonel and Faber fit into the picture.
Storing content
Colonel is a data store optimised for content storage. In basic terms, a typical CMS needs basic storage of structured data, a history of changes (audit trail) and the ability to search the content, both full-text and specific querying (e.g. find all articles with a given tag).
Colonel documents hold basic structured data, i.e. attributes and their values, like title, body, tags, etc. There is no fixed schema, so each document can potentially be different, but Colonel encourages defining document types to aid the indexing for search (you want a title to have a higher weight than the body text for instance).
Edits to a Colonel document are non-destructive. Every single update to a Colonel document creates a new revision. The revisions form a browsable timeline of edits tagged with author, date and a message, providing an audit trail of changes to each document.
Under the hood, Colonel uses elasticsearch, an enterprise grade search engine, to provide querying and full-text search in the document repository using the full power of the elasticsearch query language and additional features.
In the case of Sky Help, the CMS is only storing the article content. All media files (images and videos) are stored separately in so called content hubs. We leveraged an existing video hub built by Sky and built an image hub to store images. All of the three provide a RESTful web API so they are easy to integrate with, in order to consume the content for different channels, e.g. a mobile application.
The Help & Support site is an example of such consumer, it fetches data from the three content management systems and assembles it to a final page for the customer. That way adding features for the customer doesn’t affect the editorial staff and vice versa, unless necessary.
Publishing content
In most situations, it is not enough for content to be visible to the customers the second it is saved. There is usually some process of checking and approving the content from different perspectives (UX, technical, branding, legal, even grammar...). Therefore a publishing workflow defining the steps necessary for an article to go public is needed in most situations.
Colonel has support for any kind of publishing workflow, but doesn’t itself define any business logic around it (the business logic of publishing workflows is a separate problem, hard to solve in a generic way). It is entirely up to the application built on top of Colonel to handle the transition between workflow states, but the states themselves are built into the Colonel storage system.
Getting the documents marked as published in front of the customers is again a separate concern. In some cases this may be emailing the content out. In others uploading it to a storage system behind a content delivery network. In the case of Sky Help & Support site, the content is made available over a simple RESTful API for the user facing site to consume.
Editing content
The same argument applies to content editing tools as to the whole CMS. In fact, the editing experience is central to the whole product. And as with a CMS there is no one size fits all solution. Most systems ship with some form of HTML WYSIWYG editor and the vast majority of them are mediocre at best.
The issue is again in the definition of the problem - what does editing content mean? The ideal editing tool will be very different to edit prose content (like a blog) and to edit a site home page for example. Colonel makes no assumptions about how the content it stores is produced or edited, nor how it is going to be consumed.
For the article pages on Sky Help & Support site, the best way to build them is from elements of the Sky style guide. The elements are simple blocks of various types you can arrange in a certain order and customize. An example article would consist of a text block, an image block, another text block and an accordion block with each section again containing a text block and an image block.
This structure can be stored semantically (not as HTML), meaning when the style guide changes, you don’t need to update all the content as you would with a stock WYSIWYG HTML editor. You just change the rendering component in the user facing part of the system.
To edit this block based content, we built an editor called Faber (pronounced /ˈfa.ber/, from latin for builder or craftsman. Coming up with names is difficult!). Faber lets you build content from components while giving you an almost WYSIWYG view of the result. For some components it also lets you very easily switch the presentation option (say from accordion to tabs).
Faber is only good for building content flow from blocks. For other type of content, we would use (or build) a different editor and the CMS is built to allow different editors for different content types to be used.
Divide and conquer
By now it’s hopefully obvious that the way to software reusability isn’t through reusing complete solutions. They would need to be so generic they’d result, best case, in a “good enough” experience.
The right approach to take is “divide and conquer”, i.e. split the problem into smaller, better defined sub-problems and devise solutions for them, potentially different solutions for different instances. The final product is modular, with each module generic enough for good reuse and specific enough to perfectly fit the problem.
Building systems this way gives two ways of functionality reuse:
- derived functionality, e.g. content types created on top of the Colonel documents or specific content blocks built on top of the basic Faber block
- shared functionality, e.g. reusing the image hub to store images for different uses, even sharing the individual images across them.
Instead of installing and configuring an off-the-shelf solution, you can spend the same amount of time assembling a selection of reusable tools that fit the requirements perfectly and end up with a much superior user experience and overall product. With this approach you get the flexibility of a bespoke build and time and cost savings of using existing tools at the same time.
In fact this is the reason open source software gets so much attention, because the open source software packages are small and generic enough to be useful in many situations and sharing them and bringing improvements back into them benefits everyone who uses them. And this is precisely the reason why we ended up open sourcing Colonel and Faber.
Conclusion
There is no need to live with a sub-par user experience or reduce the creative input to merely exploring configuration options and available plugins. In the same amount of time and for the same cost, you can have a product that fits your requirements perfectly, because you have built it yourself, to fit your customers’ requirements. And instead of starting from scratch, you have been able to benefit from well designed solutions to each of the individual problems you are facing. In short, use a set of small, sharp tools that are right for the job!
Artwork by Nathalie Goepel
