Skip to main content

Publication, Part of

Report 1: Care Processes and Treatment Targets 2020-21, Full Report

Audit, Survey, Other reports and statistics

Current Chapter

Data Quality Statement


Data Quality Statement

Introduction

The National Diabetes Audit (NDA) is managed by the Healthcare Quality Improvement Partnership (HQIP) on behalf of NHS England and Improvement and delivered by NHS Digital, working in collaboration with Diabetes UK and Office for Health Inequalities and Disparities (OHID, formerly Public Health England).

The NDA is a major national clinical audit, which measures the effectiveness of diabetes healthcare against Clinical Guidelines and National Institute for Health and Care Excellence  (NICE) Quality Standards, in England and Wales. The NDA collects and analyses data for use by a range of stakeholders to drive changes and improvements in the quality of services and health outcomes for people with diabetes.

The NDA answers 5 key questions:

  1. Is everyone with diabetes diagnosed and recorded on a practice diabetes register?
  2. What percentage of people registered with diabetes received the NICE key processes of diabetes care?
  3. What percentage of people registered with diabetes achieved NICE defined treatment targets for glucose control, blood pressure and blood cholesterol?
  4. What percentage of people registered with diabetes are offered and attend a structured education course?
  5. For people registered with diabetes what are the rates of acute and long-term complications (disease outcomes)?
 

The NDA supports improvement in the quality of diabetes care by enabling participating NHS services and organisations to:

  • Assess local practice against NICE guidelines.
  • Compare their care and care outcomes with similar services and organisations.
  • Identify gaps or shortfalls that are priorities for improvement.
  • Identify and share best practice.
  • Provide comprehensive national pictures of diabetes care and outcomes in England and Wales.

Through participation in the audit, local services are able to benchmark their performance and identify where they are performing well and improve the quality of treatment and care they provide. On a national level, wide participation in the audit also provides an overview of the quality of care being provided in England and Wales.

 

Relevance

Data is extracted from general practice (GP) clinical systems and specialist diabetes services in secondary care hospitals. The audit is a voluntary audit for GP practices, meaning GP practices need to opt in to be included. For specialist services, participation in the audit is mandatory. In 2020-21, the participation rate was 99.3% of all GP practices in England and Wales.

The information collected from GP practices for the audit is individual level data and contains demographic information such as age, sex, ethnicity, diabetes duration and geographic variables such as postcode. As the data collected is identifiable data, it is subject to dissent, meaning patients can remove their consent from inclusion of their data the NDA. This means that the registrations for people with diabetes captured may not be a true reflection of the whole population of people with diabetes for that GP practice.


Accuracy and reliability

The 2020-21 audit covers the majority of England and Wales with a participation rate of 99.3%. Participation is published at Clinical Commissioning Group (CCG) and Local Health Board (LHB) level. Caution should be taken when looking at areas with low participation and when looking at areas with variable participation over time.

Primary care data is extracted from GP clinical systems via the General Practice Extraction Service (GPES); the practice must approve the extraction. Specialist diabetes services submit their data through the Clinical Audit Platform (CAP). CAP was used for the first time in the 2018-19 audit and is via a bespoke data collection. It is a continuous collection and services are given a deadline by which to submit data for inclusion in the audit. Any data submitted after that date may be included in the next audit if observation dates fall into the appropriate period.

Data has been reported at GP practice level since the 2013-14 audit.

The audit report includes only information on 8 of the 9 care processes for Wales. This is due to an issue in data collection which was identified in the 2011-12 audit. Examination of the issue highlighted inconsistencies in how eye screening was being recorded in GP records. Therefore, this care process is not covered in the audit for Wales.

There are several data quality issues the reader should be aware of when looking at the data included in this report: 

1. Urine albumin/creatinine ratio care process

Caution should be taken when using screening test results for early kidney disease (Urine Albumin Creatinine Ratio, UACR) prior to 2013-14 and assessing variation across CCGs for this care process over time. 

Some areas recorded the albumin values in ‘text’ field format that did not carry through to the audit during the extraction process. 

This method of recording is thought to be due to the different ways this test is carried out across localities. Although these values are not brought through to the dataset, the dates for these values are. This means that even if the value is not extracted, the date that the test took place is. Therefore, this date is used to determine if the care process has taken place.

Since 2013-14, data has been extracted in a consistent way across all service providers. This resulted in better performance for albumin care process completion in 2013-14. However, due to the retirement of the Quality Outcomes Framework (QOF) indicator and the potential refocusing of GP practices, performance in recording this care process and gradually fell until 2018-19 in England (and then picked up in 2019-20). In Wales the care process percentage continued to fall gradually until 2019-20. In 2020-21, a more significant fall was seen which is thought to be a result of the COVID-19 pandemic.

2. Blood pressure treatment target

A data quality issue for the blood pressure treatment target affected a small number of GP practices in 2015-16. A reading for systolic blood pressure was recorded but measurements of diastolic blood pressure were incomplete. These patients have been recorded as not meeting the treatment target and have not been included in the treatment target calculation. This issue does not affect the recording of whether the annual check for blood pressure has taken place.

3. HbA1c care process

An issue affected data supplied to NHS Digital for a large number of GP practices in the 2011-12 audit regarding HbA1c (blood glucose) recording. While this did not materially affect the findings in the national report, the CCG level care process and treatment target reports covering potentially affected practices for the 2011-12 publication have been removed from our website.  The issue is restricted to the 2011-12 audit data.

4. Possible data file issues in 2016-17 when submitted 14-16th June 2017

We are aware of an issue that may have affected a number of data files collected for the 2016-17 audit that were received between the 14th and 16th June 2017. An error occurred in transit of some files to NHS Digital and as a result some practices may show as underperforming when this is not necessarily the case. For example, BMI results for Kenyon Medical Centres (M86015) and HbA1c results for Staithes Surgery (B82046).

5. Foot surveillance data from TPP SystmOne GP clinical system 2016-17

It has been found that foot surveillance data may have been under-reported in 2016-17 for several GP practices that used the TPP SystmOne clinical system. This may account for some of the fall in overall foot surveillance completion in 2016-17.

6. Incorrect data submitted for Leicester Royal Infirmary (RWEAA) in 2016-17

It has been identified that the data submitted to the audit from Leicester Royal Infirmary was for far fewer patients than was intended. The results for Leicester Royal Infirmary in 2016-17 should be treated with extreme caution.

7. Incorrect HbA1c data submitted for Hull and East Yorkshire Hospitals NHS Trust (RWA) in 2016-17

It has been identified that the HbA1c data submitted to the audit from Hull and East Yorkshire Hospitals NHS Trust was incorrect. The data show a much lower HbA1c reading for patients than was the case. The performance of this trust appears to be much better for 2016-17 than it was, so any relative deterioration in performance should be treated with extreme caution.

8. Diabetes type in 2017-18

Initial data quality checks on the 2017-18 NDA data showed that a higher than expected number of people had a Read code which matched to a general diabetes diagnosis, rather than specifically type 1, type 2 or 1 of the smaller types. Where possible, diabetes type data collected in previous audits was used to override the general diagnosis with a more specific diabetes type. This method was also used to complete the diabetes type for individuals with missing data. Although this method of improving the data worked well, it is still likely that a small number of people with type 1 diabetes, who have not appeared in a previous audit, will have their data included in the “type 2 and other” category.

9. Body mass index (BMI) care process in 2017-18

GP practices using the Vision clinical system supplier were unable to submit a BMI date or value for people registered with the practice under the new GPES collection process, due to a technical issue. To ensure that BMI data could be included in the NDA for these practices, the 2017-18 collection also extracted height and weight data, where this was available. For all system suppliers, as well as using submitted BMI dates to indicate that the care process had taken place, dates were used where weights had been taken. This may account for some of the uplift in BMI care process completion in 2017-18.

10. Incorrect organisation code and insulin pump data submitted for Manchester Royal Infirmary (RW3MR) in 2017-18

Data for Manchester Royal Infirmary were incorrectly submitted, and subsequently reported in the NDA, under the organisation code for Manchester Children’s Hospital (RW3RM). Also, all patients were incorrectly submitted as using an insulin pump.  

11. Urine albumin data from TPP SystmOne GP clinical system in 2018-19

Urine albumin data may have been under-reported in 2018-19 for several GP practices that used the TPP SystmOne clinical system. This may account for some of the fall in urine albumin data in 2018-19.

12. Drugs data from Microtest GP clinical system in 2018-19

There were no submissions of drug data from any of the GP practices in South West England which use the Microtest clinical system in 2018-19. This particularly affected drug data for NHS Kernow CCG.

13. Creatinine SNOMED codes 2019-20 and 2020-21

There has been an issue with the SNOMED codes used to identify if a person has had their serum creatinine care process check. 2 serum/plasma creatinine codes were removed from the NDA creatinine code set during the universal SNOMED code refresh. This affected the measurement of creatinine care process completion in a small number of health economies (particularly affecting Oxford in 2019-20 and in 2020-21 only Mid-Essex CCG was affected). The issue has the potential to influence the all 8/9 care process percentages for affected organisations/areas. NDA business rules are being amended to add these codes back into future NDA data extractions.

14. Homerton University Hospital specialist service data 2019-20

Homerton University Hospital submitted retinopathy/blood pressure data into CAP for all patients receiving retinal examination.  People were identified as legitimately part of the NDA audit, and included in the audit, if additional care process information was also provided.


Testing

Relevant data is extracted from GP systems via Read/SNOMED codes. The list of codes is available upon request. The NHS Digital Primary Care Domain developed the specifications specifically for the NDA extract, and these were verified on several system types prior to initial collection of the data.


Validation

Data was automatically extracted from GP systems in England via system supplier-developed queries and GPES. Internal validation checks were completed within the NDA team to ensure that data had been received from all eligible participating practices and there were no systematic issues with the data. Comparisons were made with GP practice and CCG level counts from previous years. 

2 files per GP practice were submitted for Wales by NHS Wales Informatics Service (NWIS) for the 2020-21 audit year. The submissions were checked within the NDA team for completeness and consistency with 2019-20.

Any data quality issues with the data are included alongside analysis to ensure users can interpret the results appropriately. 

To improve alignment with NICE guidelines, a revised Read code set of terms describing digital eye screening was used. This identified that variation in the use of terminology and its impact on the consistency of data extraction from electronic clinical records rendered it unreliable as a measure of this care process. The NHS Diabetic Eye Screening Programme (NHS DESP) records every digital eye screening and we believe that its records should now be used as the preferred measure for this annual care process. Presently this is reported only nationally.
 


Timeliness and punctuality

Timeliness

The 2020-21 audit collected data covering the period 1 January 2020 to 31 March 2021. Primary care data was extracted from GP clinical systems in England in May 2021. Specialist diabetes services, other than structured education only providers, submitted their data on a continuous basis and the cut-off date for specialist services data was 4 July 2021. Any submissions prior to 4 July 2020  were assumed to have been part of the 2019-20 submission and not included. 

Structured education information providers now submit into a separated continuous collection.  The same end-date cut-off was used but as these providers had submitted their data early all submissions before this cut-off were accepted.

The initial datafile from NWIS (Wales primary care data) was received at the end of June 2021. The data was processed and ready to analyse in October 2021 and published on 9 December 2021.

Punctuality

Reports are produced and data presented at national, CCG, LHB and general practice level for England. The time lag between the end of the audit period and the publication of the data was 8 and a half months.


Accessibility and clarity

Key findings and recommendations are presented in the full NDA report published on 14 July 2021. Data was initially provided in December 2021 for England and Wales at GP, CCG and LHB levels in interactive Excel spreadsheets for ease of interrogation and further analysis by users. Web links to the technical specifications of the data are available through National Diabetes Audit Programme - NHS Digital

The key elements of the data collection are presented in the methodology document that accompanies this document.


Coherence and comparability

Comparability over time

The NDA has been running since 2003-04; however, there are inconsistencies in how the data has been processed prior to 2009-10. Therefore, caution should be taken when looking at data from earlier years of the audit.

Users should also bear in mind the differences in participation over time.

In the 2012-13 audit, the blood pressure treatment target was amended to be consistent with the Quality and Outcomes Framework (QOF). A paper was produced to show the impact of this change on the previously published data

Reduction in participation in 2013-14 and 2014-15

There was a drop in participation in the 2013-14 and 2014-15 audit collections due to changes in the way the data was collected from GP practices. There was increased complexity to the processes for registrations and submissions due to new Information Governance ‘opt in’ requirements.  The collection used to be on an opt out basis. Changes to the Confidential Advisory Group (CAG) requirements meant that from 2013-14 this changed to an opt in basis.  The new governance meant that GP practices had to actively give permission for their data to be extracted or extract the information themselves and provide it directly to the audit.

Participation of GP practices in England is variable across the country and this may be due to the varied levels of support for participation offered to GP practices by CCGs.

Analysis was completed to ensure that the data collected for 2013-14 and 2014-15 were representative both in demographics and in performance:

  • There was a change in the age profile of the cohort of patients included in the audit in these collections. Standardising results, to account for this, leads to only a very slight change in the national figures. Therefore, to simplify interpretation and explanation, the results have not been standardised.
  • With a reduction in participation there was the potential for those practices taking part to be skewed towards those that perform well. Analysis was carried out on practices that participated in the latest collections and those that participated in earlier collections. The findings suggest that there is no bias towards high performing practices taking part.

Comparability with other sources

The Quality and Outcomes Framework (QOF) collects information on people registered with diabetes. However, this is only broadly comparable as there are differences in the collection period and the definitions of the indicators.  More information on the differences can be found here.

QOF collects the number of patients aged 17 years and above with type 1 and type 2 diabetes, QOF is an aggregated return and is mandatory for GP practices to participate. The table below compares the number of diabetes registrations in the NDA with the number of diabetes registrations in QOF and shows the case ascertainment based on this.

The Quality Assurance and Improvement Framework (QAIF) was introduced as part of the General Medical Services (GMS) contract reform for 2019-20. For Wales QOF ceased after 2018-19, with QAIF introduced as its replacement.

Country  Audit year NDA registrations1 QOF/QAIF registrations3  Percentage of patients recorded in NDA compared with QOF
England 2020-21 3,475,290 3,491,868 99.50%
2019-20 3,400,195 3,455,175 98.40%
2018-19 3,254,290 3,319,265 98.00%
2017-18 3,131,775 3,196,125 98.00%
2016-17 2,872,5652 3,116,400 92.20%
2015-16 2,511,863 3,033,529 82.80%
2014-15 1,702,610 2,913,538 58.40%
2013-14 1,586,380 2,814,004 56.40%
Wales 2020-215 205,860 204,326 100.80%
2019-204 207,465 198,885 104.30%
2018-19 196,830 198,885 99.00%
2017-18 195,975 194,695 100.70%
2016-17 191,260 191,590 99.80%
2015-16 189,378 188,644 100.40%
2014-15 176,472 183,348 96.20%
2013-14 159,981 177,212 90.30%

1. NDA data is collected over a 15-month period, between 1st January and 31st March, whereas QOF data is collected over a 12-month period, between 1st April and the 31st March. Therefore, the figures are not directly comparable. NDA registrations, above, count only patients that have a GP practice record and are aged 17 year or over to be as consistent as possible with QOF.  This rationale was used for all the data shown in this table though it was not used consistently in preceding audit years.  Consequently, some figures may differ from previously published Quality Statements.

2. The 2016-17 audit had a higher proportion of people (1.56%) without an age or date of birth that are not included in this total.

3. QOF registrations represent the total number of diabetes registrations provided by GP practices participating in QOF in the year of interest. These are for people aged 17 years or older.

4. In 2019-20 QAIF data were not available for Wales at a practice level in time for publication, QOF estimates from 2018 were used in place of the 2019 values to calculate percentages recorded in NDA.

5. In 2020-21 the most recent QAIF data at the time of publication, 2019-20 (covering the period October 19-September 20), were used to calculate percentages recorded in the NDA.


Assessment of user needs and perceptions

The NDA advisory group (consisting of patient representatives, Diabetes UK, clinicians, GP representatives, researchers and interested analysts from NHS Digital and OHID) provide advice on both analysis and content of the reports as well as the direction and development of the audit.

The NDA team has an active role in the National Cardiovascular Health Intelligence Network (NCVIN) workshops to gain a better understanding of how the CCGs and localities use the data and how we can improve the publication and supporting information.  These workshops are conducted quarterly and are co-ordinated by OHID, bringing together epidemiologists, analysts, clinicians and patient representatives.

NHS Digital is keen to gain a better understanding of the users of this publication and of their needs.  Your feedback is welcome and may be sent to [email protected] (please include ‘National Diabetes Audit’ in the subject line).

Alternatively, you can call our enquiries team on 0300 303 5678.

Or write to:

NHS Digital
7-8 Wellington Place
Leeds,
LS1 4AP


Last edited: 15 April 2024 3:33 pm