Skip to main content

Data quality

The DQMI is a monthly publication about data quality in the NHS, which provides data submitters with timely and transparent information.

Data Quality Assurance

The January publication remains unchanged in terms of dataset coverage and the methodology. The main point to note is this publication contains the scores and data to support the DQMI element of the quarter three performance against the Mental Health Commissioning for Quality and Innovation (CQUINS) Scheme.

We are also pleased to announce the release of the NHS Digital Provider Data Quality Assurance Framework. The framework is published as two separate but linked documents. Part 1 is aimed at provider organisations who, in terms of data quality assurance, need to get started. Part 2 of the framework is for those that are already looking at building on their existing data quality assurance processes and practices.

Each part of the framework covers five main themes:

  • oversight
  • process
  • people
  • systems
  • measures

Within each section describes its objective, the benefits associate with it and current best practice. 

Several sections include a case study and links to supporting resources and the intention is to extend over time through a collaboration site that will be launched in February 2020.  

Each part of the framework also includes an Assurance Checklist to assist provider organisations assess where they are in terms of data quality assurance and what gaps there might be that the framework can help fill.

An electronic version of the checklist is in development and this page will be updated with a link once it is available. We will then be asking provider organisations to complete the checklist on a voluntary basis to provide insight into current data quality assurance across providers.

Finally, the framework has been developed to meet the requirements of Assertion 1.7 of the Data Security & Protection tool and support the published technical guidance. The intention is that by implementing the framework, a provider organisation will have the evidence needed to support their assessed level against Assertion 1.7.

Any questions about the above or the DQMI in general then please contact us at dqateam@nhs.net.

Current Data Quality Maturity Index (DQMI)

The DQMI is a monthly publication about data quality in the NHS, which provides data submitters with timely and transparent information.

Links to all of the DQMI publications, the associated methodology documents and the interactive report can be found below:

July 2020

Provider Data Quality Assurance Framework

The Provider Data Quality Assurance Framework is aimed at provider organisations who, in terms of data quality assurance, need to get started and those that are looking to build on their existing data quality assurance processes and practices.  

The framework covers five main themes:

  • oversight
  • process
  • people
  • systems
  • measures

Within each section describes its objective, the benefits associate with it and current best practice. 

Several sections include a case study and links to supporting resources and the intention is to extend over time through a collaboration site that will be launched in February 2020.

Each part of the framework also includes an Assurance Checklist to assist provider organisations assess where they are in terms of data quality assurance and what gaps there might be that the framework can help fill. The framework will continue to be developed and additional resources made available through this web page.

The documents can be accessed here:

The framework has been developed to meet the requirements of Assertion 1.7 of the Data Security & Protection tool and support the published technical guidance.

Providers are encouraged to complete either part 1 or part 2 of the online data quality assurance self-assessment surveys. 

Self-assessment Data Quality Assurance checklist - Part 1

Self-Assessment Data Quality Assurance Checklist - Part 2

Both surveys (part 1 and part 2) correspond to part 1 and part 2 respectively of the Provider Data Quality Assurance Framework.

Responses to the survey will assist NHS Digital to understand national data quality assurance practice and associated support requirements.  

MHSDS Validity and Integrity Measures SQL

The two pieces of code below are used by NHS Digital’s MHSDS team to generate the raw data provided to the DQ team that we then process to produce the DQMI scores and associated reports.

The code is run against the MHSDS data after the data has been submitted and processed through the DPS Portal. It will, therefore, require modification locally to be run against the pre-submission data. 

Whilst we cannot assist in this modification, we would be willing to work with a small number of Mental Health Services providers to understand the technical steps required to use the code locally and to then share this learning alongside the code.

Create validity measures SQL

Create integrity measures SQL

Current comorbidity diagnostic persistence

Diagnostic codes for conditions that do not remit should not be lost from patient records over time - this "diagnostic persistence" is a measure of data quality.

The below factsheet highlights the proportion of patient episodes that include, or omit, a code for one of five clinical diagnoses where there is reason to expect that a code should continue to be recorded.

We reviewed diagnostic codes for all episodes of hospital care over a three month period, and then looked back to compare whether a relevant code was previously recorded.

February 2018

DQMI Comorbidity Outputs 17-18 Q2 Jul to Sep v1.0 Final [859kb](Opens in a new window)

February Diagnostic Persistence Infographic_FINAL v0.1 [518kb](Opens in a new window)

Note: The analysis is to show diagnostic coding for conditions that do not remit should not be lost from patient records over time - this "diagnostic persistence" is a measure of data quality, it is not to show poor diagnosis or treatment of conditions.

Historic comorbidity diagnostic persistence

Historic DQMI publications

June 2020

May 2020

April 2020

March 2020

February 2020

January 2020

December 2019

DQMI-24 (September 2019)

DQMI-24 CSV

DQMI-24 Methodology

Power BI Interactive Report

DQMI Power BI User Guide

November 2019

DQMI-23 (August 2019)

DQMI-23 CSV

DQMI-23 Methodology

Power BI Interactive Report

DQMI Power BI User Guide

October 2019

DQMI-22 (July 2019)

DQMI-22 CSV

DQMI-22 Methodology

Power BI Interactive Report

DQMI Power BI User Guide

September 2019

DQMI-21 (June 2019)

DQMI-21 CSV

DQMI-21 Methodology

Power BI Interactive Report

DQMI Power BI User Guide

August 2019

DQMI- 20 (May 2019)

DQMI- 20 CSV

DQMI- 20 Methodology

Power BI Interactive Report

DQMI Power BI User Guide

July 2019

DQMI-19 (April 2019) (Updated 20 February 2020 to include MSDS data)

DQMI-19 CSV (Updated 20 February 2020 to include MSDS data)

DQMI-19 Methodology

Power BI Interactive Report

DQMI  Power BI User Guide

June 2019

Updated 26 July 2019 to include DID data

DQMI-18 (March 2019) (Updated 21 August 2019 with changes made to the default in excess calculation)

DQMI-18 CSV (Updated 21 August 2019 with changes made to the default in excess calculation)

DQMI-18 Methodology

Power BI Interactive Report

DQMI  Power BI User Guide

May 2019

DQMI-17 Methodology

DQMI-17 (February 2019)

DQMI-17 CSV

Power BI Interactive Report

DQMI Power BI User Guide

April 2019

DQMI-13-16 Methodology

DQMI-13 (October 2018)

DQMI-13 CSV

DQMI-14 (November 2018)

DQMI -14 CSV

DQMI-15 (December 2018)

DQMI-15 CSV

DQMI-16 (January 2019) (Updated 20 February 2020 to include MSDS Data)

DQMI-16 CSV (Updated 20 February 2020 to include MSDS Data)

Power BI Interactive Report

DQMI Power BI User Guide

DQMI-14 CSV

February 2019

DQMI-12 (2018-19 Q2) - covers data for the period of July – September 2018

DQMI-12 Methodology

DQMI 12 CSV

Power BI Interactive Report

DQMI Power BI User Guide February 2019

November 2018

DQMI-11(2018-19 Q1)

DQMI-11 Methodology

DQMI 11 CSV

Power BI Interactive Report

DQMI Power BI User Guide November 2018

August 2018

DQMI-10 (2017-18 Q4)

DQMI-10 Methodology

DQMI 10 CSV

Power BI Interactive Report

DQMI Power BI User Guide August 2018

May 2018

DQMI-9 (2017-18 Q3)

DQMI-9 Methodology

DQMI 9 CSV

Power BI Interactive Report

DQMI Power BI User Guide May 2018

February 2018

DQMI-8 (2017-18_Q2) [694kb]

DQMI-8_Methodology

DQMI_8_CSV [2Mb](Opens in a new window)

Power BI Interactive Report

DQMI Power BI User Guide February 2018

November 2017

There are missing records for a number of providers (predominantly independent sector) in the CDS (APC, A&E, OP) data. This was caused by a change introduced by SUS+ in August which resulted in these missing records.

DQMI-7_(2017-18_Q1) [689kb](Opens in a new window)

DQMI-7 Methodology [1Mb](Opens in a new window)

DQMI 7 CSV [1Mb](Opens in a new window)

Power BI Interactive report

DQMI Power BI User Guide November 2017

Note: The Analysis is to show diagnostic coding for conditions that do not remit should not be lost from patient records over time - this "diagnostic persistence" is a measure of data quality, it is not to show poor diagnosis or treatment of conditions.

August 2017

DQMI-6 (2016-17 Q4) [730kb](Opens in a new window) covers data for the period of January - March 2017.

DQMI-6 Methodology [977kb](Opens in a new window) 

DQMI-6 CSV (Opens in a new window)

Power BI Interactive report

DQMI Power BI User Guide August 2017

May 2017

DQMI-5 (2016-17 Q3) [723kb](Opens in a new window) covers data for the period of October - December 2016. 

DQMI-5 Methodology [1018kb](Opens in a new window)

DQMI-5 CSV file [1Mb](Opens in a new window) CSV reissued on 24 May 2017 due to duplicates in the original file.

Power BI Interactive report

Power BI Report user guide May 2017

February 2017

DQMI-4 (2016-17 Q2) [643kb](Opens in a new window) covers data for the period of July - September 2016

DQMI-4 (2016-17 Q2) [643kb](Opens in a new window) 

DQMI-4 Methodology [1Mb](Opens in a new window)

DQMI-4 CSV [1Mb](Opens in a new window)

Power BI interactive report

Power BI Report User Guide February 2017

November 2016

DQMI-3 (2016-17 Q1) [683kb](Opens in a new window)  covers data for the period of April - June 2016

DQMI-3 Methodology [1Mb](Opens in a new window)

DQMI-3 CSV [1Mb](Opens in a new window)

Power BI interactive report

Power BI Report User Guide November 2016

August 2016

DQMI-2 (2015-16 Q4) covers data for the period of January - March 2016

DQ_maturity_index_introduction_and_methodology__15_8_16

DQMI-2 CSV file

Power BI interactive report

May 2016

DQMI-1 (2015 Annual) covers data for the period of January - December 2015

DQMI-1 methodology document

The Health and Social Care Act 2012 (section 266) states that our statutory data quality role is to assess the extent to which the data we collect meets defined national standards and to publish the results of the assessments. In addition, we may give advice or guidance on data quality relating to the collection, analysis, publication or other dissemination of data and information.

Our Data Quality Assurance Strategy 2018-2020 sets out how we will fulfil these responsibilities.

The Health and Social Care Act 2012 (section 266) states that our statutory data quality role is to assess the extent to which the data we collect meets defined national standards and to publish the results of the assessments. In addition, we may give advice or guidance on data quality relating to the collection, analysis, publication or other dissemination of data and information.

Our Data Quality Assurance Strategy 2018-2020 sets out how we will fulfil these responsibilities.

DQA steering group

Data Quality Assurance (DQA) activity is governed by the DQA Steering Group who meet every two months to provide overall strategic direction for DQA.

The group is made up of internal members, as well as external representatives from the Care Quality Commission, Department of Health and Social Care, NHS England and NHS Improvement. We are working to ensure that the provider community are also represented at the DQA Steering Group.

The group's Terms of Reference are available for reference, and any queries about the group should be directed to: DQAteam@nhs.net.

DQA user forum

The DQA user forum meets quarterly with representatives from a number of clinical commissioning groups (CCGs), commissioning support units (CSUs), NHS England and the Care Quality Commission (CQC).

The forum is in its early stages but has already seen some very positive actions come out of the first meeting to proactively support the improvement of data quality.

DQA provider forum

A DQA provider forum meets quarterly with representatives from 20 provider organisations from around the country.

The forum is supported by a national provider E-forum enabling further engagement from over 100 active members.

Quick links

Further information

  1. external
Last edited: 3 August 2020 12:55 pm