Quality and Outcomes Framework, Achievement, prevalence and exceptions data 2018-19 [PAS]

The Quality and Outcomes Framework (QOF) was introduced as part of the General Medical Services (GMS) contract on 1 April 2004. The objective of the QOF is to improve the quality of care patients are given by rewarding practices for the quality of care they provide to their patients. The QOF is therefore an incentive payment scheme, not a performance management tool, and a key principle is that QOF indicators should be based on the best available research evidence. Participation by practices in the QOF is voluntary, though participation rates are very high, with most Personal Medical Services (PMS) practices also taking part.

The QOF contains three main components, known as domains. The three domains are:

• Clinical
• Public health
• Public health - additional services

Each domain consists of a set of achievement measures, known as indicators, against which practices score points according to their level of achievement. The indicators included in the current reporting year are detailed in the accompanying 'QOF 2018-19: Indicator definitions' file, and details of changes to indicators over time. 

QOF information from previous years has been published, and via an online search function.

In 2018-19, all the QOF data were collected from practices by the GPES. The GPES is a centrally managed primary care data extraction service that extracts information from GP IT systems for a range of purposes at a national level. The GPES relays data to the CQRS.

The CQRS has been used to calculate payments for GP practices across England for the 2018-19 financial year. The service calculates achievement and payments on quality services, including the Quality and Outcomes Framework (QOF), as well as Enhanced Services (ESs) and some other clinical services (e.g. vaccinations and immunisations).

PMS practices are able to negotiate local contracts with their commissioning organisations for the provision of all services. PMS practices may also participate in the QOF, and they may either follow the national QOF framework or enter into local QOF arrangements. PMS practices with local contractual arrangements are included in the published QOF information, and in the figures presented in the report.

Where PMS practices use the national QOF, their achievement (in terms of the 559 QOF points available) is subject to a deduction (approximately 100 points) before QOF points are turned into QOF payments. This is because many PMS practices already have a chronic disease management allowance, a sustained quality allowance and a cervical cytology payment included in their baseline payments. GMS practices do not receive such payments, but receive similar payments through the QOF. To ensure comparability between GMS and PMS practices, the QOF deduction for PMS practices ensures that they do not receive the same payments twice. Because the report covers QOF achievement and not payments, all QOF achievement shown is based on QOF points prior to PMS deductions. This is to allow comparability in levels of achievement – so that where GMS and PMS practices have maximum QOF achievement, both are regarded as having achieved the maximum 559 points.

There are no patient-specific data in CQRS because this is not required to support the QOF. For example, GPES captures aggregate information for each practice on patients with coronary heart disease and on patients with diabetes, but it is not possible to identify or analyse information about individual patients. It is therefore not possible to identify the number of patients with both of these diseases.

QOF data for this release were downloaded on 2 July 2019, and so include all adjustments up to 1 July 2019.

Following validation (see below), the published QOF dataset includes data for 6,873 practices. In the 2018-19 reporting year 7,227 practices eligible for QOF were open and active at some point; this yields a coverage of 95.1 per cent.

Each year, when the data have been extracted from CQRS and processed, NHS Digital invites NHS England regional local offices to view the total QOF points for each of their practices, as held on CQRS at the point of extraction for the QOF publication. Named contacts from the regional local offices can log in to an online system, check the achievement data for the practices in their area, and provide an explanation of any relevant issues.

294 practices were removed from the final dataset as a result of this validation exercise. 32 practices were not explicitly commented on during the validation process; these practices were listed as closed or dormant at the end of the reporting year and scored either zero or 14 points, and were removed on advice of the Responsible Statistician. The remaining 262 practices were removed at the request of the validators. All comments made during the validation exercise are available to view in the file QOF 2018-19: Practice validation comments file, which can be accessed on the publication homepage.

The 2018-19 QOF information published by NHS Digital includes practice list sizes as at 1 April 2019. In the context of this publication, these list sizes are used as the basis for the calculation of raw clinical prevalence.

Prior to 2015-16, practice list sizes as at 1 January of the reporting year were used. These figures are still used in CQRS for list size adjustments in QOF payment calculations.

The sum of the practice list sizes for the practices included in the QOF publication is 59,386,096. This number may contain duplicate patients where a given practice has closed during the financial year, as illustrated below:

In the example, Practice A’s QOF data are representative of their position at the point the practice closed, whereas Practice B’s QOF data reflect their status at the end of the financial year. As some of Practice A’s patients may have moved to Practice B when Practice A closed, they may be counted once in Practice A’s data and again in Practice B’s data. It is therefore impossible to express coverage as a percentage at patient level.

Practices have been mapped to their respective CCGs, STPs, regional local offices and regions using reference data current at 1 April 2019. This mapping has been applied to data for both the current and previous reporting year; this should be borne in mind when making comparisons between years (please see the ‘Comparing QOF data over time’ section).

There is a distinction between:

• Numbers of patients on disease registers for QOF indicator groups
• Numbers of patients relevant to specific indicators within these indicator groups
• Numbers of patients relevant to specific indicators who are included in the indicator denominator when measuring QOF achievement

Most indicator groups have an associated disease register (e.g. the atrial fibrillation indicator group is based on a register of patients with atrial fibrillation). Some conditions do not have a disease register (e.g. the blood pressure indicator group is based on a count of those who have had their blood pressure taken, which is not a disease register). The information systems which underpin the QOF hold the numbers of patients on each of these registers, for each participating practice. For example, there is a register count for all people diagnosed with atrial fibrillation at each practice.

Indicator denominators are the numbers of patients from the appropriate disease register who are counted for QOF achievement against a specific QOF indicator. (The indicator numerator is the number of those in the denominator who meet the specific indicator success criteria.)

Differences between an indicator denominator and the number on a register can be due to indicator definition. Patients who are on the disease register, but not included in the indicator denominator for definitional reasons, are referred to as exclusions. For example, some indicators refer to subsets of patients on a disease register e.g. they may only to patients who smoke. In this instance, any non-smoking patient on the disease register is excluded.

Differences between an indicator denominator and the number on a register not due to indicator definition, but rather due to individual circumstances, are referred to as exceptions. Exceptions relate to patients who are on the disease register, and who would ordinarily be included in the indicator denominator. However, they are excepted from the indicator denominator because they meet at least one of the SFE exception criteria (these are detailed below under 'Background to exception reporting') .

The normal relationship between registers, denominators, exclusions and exceptions is therefore:

\(Register = Denominator + Exclusions + Exceptions\)

Reference to ‘QOF achievement’ often refers to the percentage of available QOF points achieved. If a practice achieves the full 559 QOF points, it has achieved 100 per cent of the points available and may be said to have 100 per cent achievement across the whole QOF.

The level of achievement for certain elements of the QOF can be expressed in the same way. A practice achieving all QOF points available for indicators in the clinical domain can be said to have 100 per cent clinical achievement even though it may not have 100 per cent achievement overall.

Practices achieve the maximum QOF points for most indicators (especially clinical indicators) when they have delivered the maximum threshold to achieve the points available.

For many indicators a practice must provide a certain level of clinical care to 90 per cent of patients on a particular clinical register to achieve the maximum points.

Underlying achievement (net of exceptions) data are provided in the spreadsheets associated with the report. Since a practice can deliver the required care to fewer than 100 per cent of its patients (often around 90 per cent) to achieve the full (100 per cent) points available, there is an important distinction between percentage achievement in terms of QOF points available and the underlying achievement (net of exceptions) for specific indicators.

Underlying achievement (net of exceptions) presents the indicator numerator as a percentage of the denominator and is calculated thus:

\(\text{Underlying achievement net of exceptions} = {\text{Indicator numerator} \over \text{Indicator denominator}} * 100\)

Underlying achievement (net of exceptions) does not account for all patients covered by an indicator, as it takes no account of “exceptions” (patients to whom the indicator applies, but who are not included in the indicator denominator according to agreed exception criteria). Percentage of patients receiving the intervention gives a more accurate indication of the rate of the provision of interventions as the denominator for this measure covers all patients to whom the indicator applies, regardless of exception status (i.e. indicator exceptions and indicator denominator). This measure is calculated as follows:

\(\text{Percentage of patients receiving the intervention} = {\text{Indicator numerator} \over \text{Indicator denominator + indicator exceptions}} *100\)

Percentage of patients receiving the intervention figures are not covered in the main report, but are presented in the Prevalence, Achievements and Exceptions workbooks at national, regional, regional local office, STP, CCG and practice level.

It is not always possible for practices to achieve all of the points in the QOF. Therefore, NHS Digital produces a further measure of practice achievement. This measure takes account of instances where practices cannot achieve points because they have no patients pertinent to an indicator, and can be found in the Achievement workbook at practice level.

For example, in 2018-19 there are 559 points available in the QOF and 45 of these points are allocated to asthma indicators. If a practice does not have any patients on their asthma register, then they would be unable to achieve any of the points allocated to the asthma indicators. Therefore, even if the practice achieved all the other points available they would only be able to attain 91.9 per cent overall achievement (514 points achieved/ 559 points available)

In these circumstances, the standard ‘points achievement’ measure can be misrepresentative and may result in a practice’s achievement apparently declining from one year to the next where they have patients on a register in one year but none in the next year.

To represent practice points achievement more fairly, NHS Digital calculates adjusted maximum points achievable for each practice, effectively removing points from the calculation denominator where both of the following conditions apply:

• the practice does not have any patients in the indicator denominator
• the practice has reported no exceptions for the indicator denominator

In essence, the indicator denominator plus indicator exceptions must equal zero. This ensures maximum points achievable are not adjusted where there are patients on the relevant disease register (exceptions are included in the disease register, but not in the relevant denominator), who have not received the interventions.

For the example outlined above, for a practice with no patients on their asthma register the practice’s maximum points available would be 514 (559 points minus the ‘unachievable’ 45 asthma points). In this case, the difference between the practice’s ‘points achievement’ and ‘points achieved as a percentage of QOF points available’ would be as follows:

Points achievement = (Points achievement / All QOF points) x 100

= (514 / 559) x 100

= 91.9 per cent

Points achieved as percentage of points available = (Point achievement / QOF points available) x100

= (514 / 514) x 100

= 100 per cent

'Points achieved as a percentage of QOF points available’ figures are calculated for overall achievement and can be found in the Achievement workbook.

It is important to emphasise that QOF registers are constructed to underpin indicators on quality of care, and they do not necessarily equate to prevalence as may be defined by epidemiologists. For example, prevalence figures based on QOF registers may differ from prevalence figures from other sources because of coding or definitional issues. It is difficult to interpret year-on-year changes in the size of QOF registers, for example a gradual rise in QOF prevalence could be due partly to epidemiological factors (such as an ageing population) or to increased case finding and recording. Other factors in interpreting information on specific registers include:

• Some clinical areas have ‘resolution codes’ to reflect the nature of diseases. Others, such as the cancer register, do not;
• Some indicator groups for which there is a disease register are based on a specific age group (see table below). Prevalence for these indicator groups are calculated using a sub-set of the patient list size relating to the equivalent age group.

• Many patients are likely to suffer from co-morbidity, i.e. are diagnosed with more than one of the clinical conditions included in the QOF clinical domain. Robust analysis of co-morbidity is not possible using QOF data because QOF information is collected at an aggregate level for each practice; there are no patient-specific data within CQRS. For example, CQRS captures aggregated information for each practice on patients with coronary heart disease and on patients with diabetes, but it is not possible to identify or analyse patients with both of those diseases.
• The QOF register for ‘cardiovascular disease – primary prevention’ does not count the number of patients with cardiovascular disease. Rather, this is a register of patients with a new diagnosis of hypertension, excluding those with pre-existing coronary heart disease (CHD), diabetes, stroke/transient ischaemic attack (TIA), PAD, familial hypercholesterolemia or CKD aged 30-74.
• To be on the asthma register, patients need a diagnosis of asthma and a prescription for an asthma drug within the year.
• To be included in the obesity register a patient must be 18 or over and have a record of a BMI of 30 or higher in the previous 12 months. This requirement results in the prevalence of obesity in the QOF being much lower than the prevalence found in the Health Survey for England and other surveys.
• To be on the depression register, the patient must have a new diagnosis of depression in the preceding year that has been reviewed within given timescales.
• The QOF register for ‘Osteoporosis – secondary prevention of fragility fractures’ is a register of patients aged over 50-74 with a record of a fragility fracture since 1 April 2012 and a diagnosis of osteoporosis confirmed by a DXA scan; and those aged 75 and over with a record of a fragility fracture since 1 April 2014 and a diagnosis of osteoporosis (scan not required).

The number of patients on registers of indicators in the clinical domain can be used to calculate recorded disease prevalence, expressing the number of patients on each register as a percentage of the number of patients on practices’ lists, as described below:

\(\text{Disease prevalence} = ({\text{Number of patients on clinical register} \over \text{Number of patients on practice list}}) * 100\)

Where age-specific registers are used, disease prevalence can be calculated as:

\(\text{Disease prevalence} = ({\text{Number of patients on clinical register} \over \text{Number of patients in relevant age band on practice list}}) * 100\)

Exception reporting rates reflect the percentage of patients who are not included when determining QOF achievement and are presented for applicable indicators in the QOF. For the NHS Digital QOF publication, there is a distinction between patients who are actually exception-reported, and those whose non-inclusion in an indicator denominator is for definitional reasons (‘exclusions’).

Patients can be excepted for a number of reasons and are usually the result of a patient or a GP decision at a personal level.

Examples of exceptions could be patient or carer refusal of treatment, a patient cancels or does not attend a consultation appointment, or a GPs advice that two types of medication or treatment methodology should not be administered simultaneously.

Exceptions are only measured at indicator level, not condition level, as a patient could be excepted from more than one indicator within a condition, but would be counted more than once if these exceptions were summed.

These are usually as a result of the type of patient, and can be considered as non-inclusion in a denominator due to indicator definition. For example, all men are excluded from the cervical screening indicator, which is a female only measure. These affect the denominator, and are not shown in this publication.

Patient exception reporting applies to those indicators in the QOF where level of achievement is determined by the percentage of patients receiving the specified level of care. The GMS contract SFE includes the following:

“The QOF includes the concept of exception reporting. This has been introduced to allow practices to pursue the quality improvement agenda and not be penalised, where, for example, patients do not attend for review, or where a medication cannot be prescribed due to a contraindication or side-effect.

The following criteria have been agreed for exception reporting:

A) patients who have been recorded as refusing to attend review who have been invited on at least three occasions during the preceding twelve months

B) patients for whom it is not appropriate to review the chronic disease parameters due to particular circumstances e.g. terminal illness, extreme frailty

C) patients newly diagnosed within the practice or who have recently registered with the practice, who should have measurements made within three months and delivery of clinical standards within nine months e.g. blood pressure or cholesterol measurements within target levels

D) patients who are on maximum tolerated doses of medication whose levels remain sub-optimal

E) patients for whom prescribing a medication is not clinically appropriate e.g. those who have an allergy, another contraindication or have experienced an adverse reaction

F) where a patient has not tolerated medication

G) where a patient does not agree to investigation or treatment (informed dissent), and this has been recorded in their medical records

H) where the patient has a supervening condition which makes treatment of their condition inappropriate e.g. cholesterol reduction where the patient has liver disease

I) where an investigative service or secondary care service is unavailable.

In the case of exception reporting on criteria A and B this would apply to the disease register and these patients would be subtracted from the denominator for all other indicators. For example, in a practice with 100 patients on the CHD disease register, in which four patients have been recalled for follow-up on three occasions but have not attended and one patient has become terminally ill with metastatic breast carcinoma during the year, the denominator for reporting would be 95. This would apply to all relevant indicators in the CHD set.

In addition, practices may exception-report patients relating to single indicators, for example a patient who has heart failure due to left ventricular dysfunction (LVD) but who is intolerant of ACE inhibitors could be exception-reported. This would again be done by removing the patient from the denominator.

Practices should report the number of exceptions for each indicator set and individual indicator. Exception codes have been added to systems by suppliers. Practices will not be expected to report why individual patients were exception-reported. Practices may be called on to justify why they have excepted patients from the QOF and this should be identifiable in the clinical record.”

For each indicator in the clinical domain, the exception rate is calculated as follows:

\(\text{Exception rate} = ({\text{Number of exceptions} \over \text{Number of exceptions + indicator denominator}}) * 100\)

Therefore the recorded number of exceptions is expressed as a percentage of the number of patients on a disease register who were qualified to be part of the indicator denominator – i.e. were not counted as exclusions for definitional reasons.

A small number of practices who participate in the QOF make manual submissions to CQRS or are otherwise unable to make an electronic submission of exceptions data. For this small number of practices, no exceptions data are available. However, in order to maintain consistency with the report annexes, which are based on aggregated data from individual practices, they are included in the overall exception calculations.

This has the impact of slightly reducing the exception rates (because there are no indicator exceptions for these practices in the calculation numerator, but their indicator denominator data are included in the calculation denominator). The impact of this is minimal with an impact of approximately 0.3 per cent at practice level.

Information captured by CQRS relating to exceptions and exclusions cannot be amended on the CQRS system. CQRS is primarily a system to support QOF payments, and exception reporting is recorded as part of that process. CQRS was not designed to deliver specific management information about exception reporting, but does allow summary information on the levels of exception reporting to be generated. This information is the basis for this publication, and is presented at CCG level in the exceptions and exclusions at CCG level file.

CQRS does not allow a presentation of exceptions broken down by each of the nine Statement of Financial Entitlements (SFE) exception criteria outlined above. There are three reasons for this:

• CQRS uses an internal set of exception ID codes that do not map directly into the nine exception reporting criteria in the SFE; rather, these exception ID codes relate to exception reporting coding ‘clusters’ within QOF business rules, often specific to individual QOF indicators. Fewer than nine of the criteria in the SFE may apply to an indicator.
• CQRS reporting functionality does not make a distinction between exception reporting and definitional exclusions – both types of omission from indicator denominators are included on reports available to CQRS users.
• Any individual patient can be associated with more than one of the exception criteria, but only one such reason needs to be identified in order to exception-report a patient from inclusion in the indicator denominator. Only the first reason identified by the system is therefore captured.

Testing of data extraction from CQRS in line with QOF business rules on patient exceptions is primarily focused on ensuring that data values used for achievement calculations are accurate for payment purposes, i.e. that patients are not included in indicator denominators where appropriate in terms of the business rules. Therefore any testing of the order of sequencing (i.e. the order whereby systems check for different exception codes or criteria) is secondary. Different GP clinical information systems may follow different sequencing without this impacting on payment accuracy.

The CQRS system was established as a mechanism to support the calculation of practice QOF payments. It is not a totally comprehensive source of data on quality of care in general practice, but it is potentially a rich and valuable source of information for healthcare organisations, analysts and researchers, providing the limitations of the data are acknowledged.

Levels of QOF achievement will be related to a variety of local circumstances, and should be interpreted in the context of those circumstances. Users of the published QOF data should be particularly careful in undertaking comparative analysis.

The following points have been raised by local healthcare organisations in consultation with NHS Digital:

Prevalence and achievement

• The ranking of practices on the basis of QOF points achieved, either overall or with respect to areas within the QOF, may be inappropriate. QOF points do not reflect practice workload issues (for example around list sizes and disease prevalence) – that is why practices’ QOF payments include adjustments for such factors.
• Comparative analysis of practice-level or CCG-level QOF achievement (or prevalence) may also be inappropriate without taking account of the underlying social and demographic characteristics of the populations concerned. The delivery of services may be related, for example, to population age/sex, ethnicity or deprivation characteristics that are not included in QOF data collection processes.
• Information on QOF achievement, as represented by QOF points, should also be interpreted with respect to local circumstances around general practice infrastructure. In undertaking comparative or explanatory analysis, users of the data should be aware of any effect of the numbers of partners (including single handed practices), local recruitment and staffing issues, issues around practice premises, and local IT issues.
• Users of the data should be aware that different types of practices may serve different communities. Comparative analysis should therefore take account of local circumstances, such as numbers on practice lists of student populations, drug users, homeless populations and asylum seekers.
• Analysis of co-morbidity (patients with more than one disease) is not possible using QOF data. QOF data are collected at an aggregate level for each practice. For example, CQRS captures aggregated information for each practice on patients with coronary heart disease and on patients with asthma, but it is not possible to identify or analyse patients with both of these diseases.
• Information held within CQRS, and the source for the published tables, is dependent on diagnosis and recording (case finding) within practices using practices’ clinical information systems.
• Measuring the quality of care is not a simple process. Within the clinical domain, the QOF does not cover every clinical condition, and only describes some aspects of the care for the clinical areas that are included. However, the QOF does provide valuable information (for example on prevalence, cholesterol levels and blood pressure) on a scale unavailable before 2004-05, and provides a measure of improvement in the delivery of care.

Exceptions

An important aim of the Quality and Outcomes Framework is to encourage appropriate and high quality clinical care for key long-term conditions. Potentially, exception reporting could influence the level of financial reward to practices.

The availability of high level information on exception reporting provides an indication of the variations in exception rates that are found between specific indicators, and between NHS organisational areas.

It is important to emphasise some of the limitations of the available data, as described previously in this document. These include practices missing from the analysis; the derivation of exception counts; and the potential for amendments to indicator denominators not mirrored by changes to counts of exceptions.

Additionally, care should be taken when interpreting high level analysis in the context of local primary care service delivery, notably in terms of the numbers of patients associated with relatively high or low exception rates. CCGs will have access to more detailed local information, and knowledge of local circumstances, to enable unusual levels of exception reporting to be investigated further.

The workbooks present data from both the current reporting year and the previous reporting year. The aggregated (i.e. non-practice level) figures presented for 2017-18 in this release will not match those published in 2017-18, as all figures have been recalculated using practice level data that can be mapped to current NHS geographies.

Year on year, there may be changes to indicator definitions, the number of points available per indicator and the number of indicators. There were no such changes between the 2015-16, 2016-17 and 2017-18 reporting years, but all changes between previous reporting years are summarised in the accompanying FAQ section. 

The clinical codes used to define the Learning Disabilities register changed significantly for QOF 2018-19, meaning the register (and associated prevalence) for 2018-19 is not comparable with previous years. 
Therefore, the register and associated QOF prevalence for 2017-18 are not included in the 2018-19 release.