For the past three years, I have split my week between seeing patients as a child and adolescent psychiatrist and working as the senior clinical lead for data at NHS Digital. This has given me a unique perspective on how two fantastic teams help patients. I have sometimes found it hard to persuade my team at NHS Digital how much impact their work has on patients.

The truth is that analysts, data scientists and the other professionals who support them play a crucial role in making it possible for clinicians to work effectively.

The NHS has some of the richest health data assets in the world. These are used by a range of people – researchers, doctors, government bodies, charities – to improve our health. People don’t always realise that getting the right data, in the right place, at the right time, can have just as much impact as a prescription or a medical procedure.

In the UK, we probably have the world’s longest and proudest tradition of using data to improve health. One of our earliest and most effective clinical informaticians was Florence Nightingale. Her examination of the causes of death of British soldiers stationed in Crimea and her highly effective deployment of this data and analysis revolutionised practice. She helped create the modern hospital. Her suggested changes in care, including mandatory hand-washing, are still critical today.

Data extracted from many records and analysed at both national and local levels helps us to understand more about health and disease, improve public health through targeted interventions, monitor safety, and plan and deliver health and social care services more effectively. For this to happen, data needs to be collected in a high quality (accurate and complete), consistent (comparable), safe and legal way.

Nationally, NHS Digital is the safe-haven for patient data and is legally accountable for the collection, storing and use of patient data for research on behalf of the health and care system. We publish over 200 statistical reports a year.

We conducted a Clinical Review to discover how people were using our data (ranging from the number of emergency hospital emissions through to patient reported outcome measures) and what impact that had on patients as well as the health and care system. 

We have not always been good at measuring the impact and we had never conducted a review of this type before.  We were therefore keen to do so and examine the feedback to see how we could improve what we are doing.

We discovered, from a sample of organisations who had accessed our data in 2018, five categories of impact:

1. Improving health and care
2. Supporting research activities
3. Supporting policy and planning
4. Supporting assessments and efficiencies
5. Improving data capabilities

Some examples of work covered by the review included:

Sophisticated use of data is set to become more and more central to the organisation and development of our health system. 

Our new Data Processing Services (DPS) platform, which went live last year, can manage larger volumes of data faster than ever before. Cloud technology makes it more secure and allows a greater range of providers to submit their data to us. It eliminates burdens on data providers by linking data sets and ensuring they don’t have to submit the same information for more than one collection.

It also allows us to create a richer and broader data set because the consistent de-identification process (ensuring patients’ identities are not revealed) lets us link data across different care settings and geographical boundaries. 

We make anonymised data available through downloads, dashboards and sector specific data hubs, such as the General Practice Data Hub. We now have over 50 interactive dashboards on our website which are open and free for anyone to access. There is a vast array of data available and our biggest challenge is being aware and focussed on what the needs of our users are. 

We have therefore worked hard with user researchers to ensure they are easy to understand for everyone. 

In future we will enable access through application programming interfaces (APIs) to make the data more easily accessible by other applications.

At the same time, our data science team are working to enhance routine analysis and will build partnerships with others, such as the Alan Turing Institute, to enable advanced analytics across the system.

One aspect of this work concerns using machine learning techniques to build predictive models There is an increasing awareness of the potential of Artificial Intelligence (AI) to transform care and an NHS AI lab was launched in August last year by NHSX. 

For example, we are looking at how we can help predict non-attendance at out-patient appointments. This would enable trusts to optimise use of scarce resources and follow-up with patients, so they get the care they need when they need it.

We are also exploring machine learning to extract medical knowledge within free-text notes. These are notes that are usually typed or dictated by clinicians and although obviously useful to the practitioners, are less suitable for secondary use by researchers who want to track trends and understand demand on a wider scale.

Many organisations are seeing data driving more proactive and personalised services. The UK Health Data Research Alliance was created to unite expertise and to ensure there is an ethical and consistent approach to the use of data, that it is of high quality and that patients trust us to use their data responsibly. 

The Alliance members include NHS England, NHS Digital and Public Health England as well as NHS Trusts, charities and professional bodies. The goal is clearly to build a more innovative health and care system that is trusted and secure and works for us all.

The full clinical review report is available on the NHS Digital website.