Skip to main content
Blog

Dr Seuss and engineering in a pandemic

Andrew Blundell, Head of Software Engineering in NHS Digital’s Product Development directorate, explains why we must split apart large systems into smaller, loosely coupled components to cope with a wild and unpredictable world – and how the NHS Digital software engineering quality framework helps us.

Dr Seuss once said: “Life’s a Great Balancing Act”.

Software engineering is like life in that respect. As software engineers, we manage many competing concerns – security, reliability, performance and so on – and great engineering depends on getting the right balance between them.

It is also about being able to respond and find a new balance when things change – and doing this so smoothly that nobody notices it is happening.

Andrew Blundell sitting at his desk at home.

Engineering in the grip of COVID-19

In 2020, we have experienced rapid and extreme context shifts in NHS software engineering. Two concerns, in particular, have become more important for us during the pandemic than ever:

  1. Scalability: Coronavirus has generated exceptionally high load on many of our services, in some cases orders of magnitude greater than before.
  2. Flow (in other words, the rate at which software changes can be delivered): The pace of business change within the NHS in 2020 has been phenomenal, and our job as engineers is to make sure that the technology doesn’t slow the business down.

For example, use of the 111 online service has has been up to 95 times higher than the baseline level during the pandemic. We’ve had to deliver software changes faster than ever to meet the demands of this rapidly changing environment and great engineering has underpinned our success in meeting these challenges.

Scalability and flow, and the techniques to achieve them, have little in common, except that both are massively challenging if you need to apply them to products that weren’t engineered with these things in mind. They are best considered from the start.

Big systems and big problems are just unacceptably complex if we are to meet the demands of our time

It’s difficult to find time to invest in improving these things when there are real and urgent business needs that must be met quickly, and the temptation is always to come back to these things ‘when there’s time’. We’ve all done this, in the name of pragmatism and with the best of intentions, and there’s almost never a ‘good time’ later. Then, something like the pandemic happens.

The good news is that the new NHS Digital software engineering quality framework gives us a huge head-start, setting out guiding principles, practices and patterns that help us to address the wide variety of modern engineering concerns, including observability, maintainability, testability, and so on.

In this blog we will focus just on the framework’s guidance on scalability and flow.

Scalability: use of managed services especially public cloud.

Flow: automation, especially for testing, and working incrementally.


The advantages of component-based design

One technique that helps with both scalability and flow is component-based design, also known as ‘loose coupling’. Using this technique, we construct systems as sets of components, sometimes known as microservices, each of which has a single job to do, known as the ‘domain’ of the component. The components handle everything needed to do their one job, including data storage, business logic, and user interfaces.

These components work in isolation, interacting with other components only via defined APIs (application programming interfaces). The limits of their isolation are called their ‘domain boundary’. They take no interest in the broader business context of why they’re being asked to perform their job: they are only interested in performing that one job.

There are a number of benefits of component-based designs, and some of these help with the particular concerns we have been discussing.

Scalability:
  • large systems often have one or more elements that are problematic to scale, for example a troublesome data-persistence layer that can scale vertically (we can make it bigger) but can’t scale horizontally (we can’t introduce more copies of it, to spread the load). When those large systems are split into multiple smaller components, that scaling challenge typically only affects a subset of those components, and the rest are easier to scale
  • component-based designs allow us to choose the best tools for each job. Technologies that are ideally suited to the small job in hand are easier to work with, perform better and can more easily handle high volumes of load
  • if a technology isn’t doing what we need (for example, handling the loads we need to manage), it’s far easier to swap it for a different technology if we’re only talking about a small component. This lowers the barrier to experimentation and innovation
Flow:
  • software changes within small components are easier to reason about, develop, and test. Every stage of the development process is simpler and quicker, which promotes higher flow
  • multiple components make it easier to run development work in parallel. For example, multiple teams can safely work in parallel if each team is working on different components, and each component has a robust domain boundary. This reduces friction and hand-offs between those teams, which again promotes higher flow

The bottom line is that component-based designs make life easier because a set of small problems is easier than one big problem. Big systems and big problems are just unacceptably complex if we are to meet the demands of our time. Simplicity is our friend.

Our world has changed. It’s wilder and more unpredictable than ever. We need to accept this and make 2021 the year we split apart our remaining large systems into smaller, loosely coupled components. There will never be a good time to do this, but getting our engineering right has never been more important.

Dr Seuss also said this: “Sometimes the questions are complicated and the answers are simple.” He’d have made a brilliant software engineer.


Get involved

We have open-sourced the NHS Digital software engineering quality framework and it is completely public for anyone who would like to use it.

The framework will continue to be enhanced, and all contributions are welcome:

  • you can suggest and comment on ideas and features in the issue list
  • you can propose changes by submitting a pull request

Interested in working at NHS Digital? Search our latest job opportunities.


Author


Last edited: 22 December 2021 2:06 pm