Patient Reported Outcome Measures (PROMs) in England, Provisional 2023/24 data

This page aims to provide users with an evidence-based assessment of the quality of the statistical output of the Provisional Patient Reported Outcome Measures publication for April 2023 to March 2024 data by reporting against those of the nine European Statistical System (ESS) quality dimensions and principles appropriate to this output.

More information regarding the background to PROMs and information relating to data quality measures can be found in the files via the "Resources" section (this can be found on the "Overview" tab).

Data quality notes for this publication

The use of PROMS data are covered by consent which specifies what the data can be used for, what other datasets (such as HES) that can link to PROMs and how long the data can be kept. This includes retaining the patient identifiers (NHS number, date of birth and postcode) for no longer than 24 months after completion of the pre-operative questionnaire. The timing of data processing means that the consent update process has already been applied to some PROMS data from April to November 22, and as a result fewer PROMS records may be found to be linked to HES hip and knee replacement activity for those months. The months most affected are October 22 followed by September and November. 

While the PROMS processing has been paused, NHSE has not sought clinical advice on maintaining the clinical procedure codes used to match PROMS data to HES using the published methodology. Specifically, any relevant changes from OPCS 4.9 to OPCS 4.10 effective from April 2023 may not have been considered. 

The range of providers in scope for PROMS will change over time, through contract changes, trust mergers etc. We do not have a definitive picture of the providers in scope in recent years but can see that some providers who submitted data in pre-pandemic were no longer submitting data for the years we are about to publish. The pause in engagement may have reduced the number of providers submitting data and the quality and completeness of the data they submitted. Please see the DQ file for more information.

Impacts of COVID-19

The following information appeared in the 2021/22 publication and will still be relevant.

In order to respond to the challenges posed by the coronavirus pandemic NHS hospitals in England were instructed to suspend all non-urgent elective surgery for patients for parts of the 2020/21 reporting period. A reduced service  continued during the 2021/22 reporting period. This has directly impacted upon reported volumes of activity pertaining to Hip & Knee replacements reported in PROMS. In addition it is possible that behaviours around activities relating to the completion, return and processing of pre and post-operative questionnaires may have also been impacted when compared to earlier years data where behaviours and processes related to managing the current pandemic were not in place.

Health providers and commissioners use PROMs publications and statistics to improve the quality of health care offered to patients. Patients (and others involved in managing patient care such as GPs and carers) can use PROMs data to help decide where to receive treatment using the interactive publication tools.

Academic researchers use these data to inform research on PROMs-eligible procedures. Patients, carers and other organisations that support patients (including GP practices and charities) may also find these statistics helpful in making informed choices about elective procedures and providers.

NHS England is keen to gain a better understanding of the users of this publication and of their needs; feedback is welcome and may be sent to [email protected] (please include ‘PROMs’ in the subject line).

Accuracy is the proximity between an estimate and the unknown true value. Reliability is the closeness of early estimates to subsequent estimated values.

Timeliness refers to the time gap between publication and the reference period. Punctuality refers to the gap between planned and actual publication dates.

Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.

The HES dataset used in the PROMs publication has been collected primarily for official administrative purposes. Information about the administrative source and its use for statistical purposes is included in NHS England’s Statement of Administrative Sources.

This publication is being made available on the world-wide-web as a combination of web pages and downloadable reports and data files. The publication may be requested in large print or other formats through the NHS Digital’s contact centre: [email protected] (please include ‘PROMs’ in the subject line).

Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain.

Other official statistics published by NHS England that report on extracts of HES inpatient data allocate episodes to time periods based on episode end date. PROMs publications, however, use the episode start date to assign records to time periods, as this date more closely represents the date of the PROMs operation.

A small number of HES episodes record more than one eligible PROMs procedure. PROMs publications report on eligible procedures, whereas other publications from HES data report on episodes. In practice, this means that if a patient undergoes both a hip and knee replacement in a single hospital episode, their hospital records this as one episode for both procedures. However, the distinction between the two is made within the PROMs analysis of the linked PROMs questionnaires.

The last provisional publication was released in July 2023. Given the delay in this publication, it may include questionnaires returned later than the six month cut off allowed for the previous reports.

To report feedback and suggestions about this publication, please email our enquiries mailbox at [email protected] (please include ‘PROMs’ in the subject line).

The PROMs programme invites all NHS-funded patients going through a PROMs-eligible procedure to complete pre- and post-operative questionnaires, which are linked, where possible, to administrative data about their procedure-related health care.

Pre-operative questionnaires may be administered on the day the patient is admitted for treatment or at a pre-assessment clinic beforehand. Paper-based questionnaires are predominantly used, but questionnaires can be delivered electronically. Typically, the questionnaire will take around 5 - 10 minutes to complete. These questionnaires are collected by providers after completion and sent to their data supplier. The data supplier then submits the pre-operative data as record-level xml files to NHS England.

Where pre-operative questionnaires link to a HES episode, NHS England sends the data supplier a file containing the PROMs Serial Number, the procedure, and the operation date. This lets data suppliers calculate when to send post-operative questionnaires to patients. Post-operative questionnaires should be sent out six months after the procedure date. Data suppliers can send out up to two follow-up mailings to patients that have not returned post-operative questionnaires, encouraging them to do so. Patients return completed post-operative questionnaires to the data supplier using a pre-paid envelope. The data supplier then submits the post-operative data to NHS England.

On receipt of the data, NHS England performs a number of validation checks and the data becomes available for NHS England analyses and publications. NHS England links pre-operative questionnaires with administrative data about patients’ hospital stays (for relevant procedures) held in the Hospital Episode Statistics (Admitted Patient Care) dataset.

The use of PROMs data are covered by consent which specifies what the data can be used for, what other datasets (such as HES) that can link to PROMs and how long the data can be kept. This includes retaining the patient identifiers (NHS number, date of birth and postcode) for no longer than 24 months after completion of the pre-operative questionnaire.

Data are subject to disclosure control before being released. More information about the disclosure control rules can be found in the Footnotes located in CSV Data Pack accompanying this report.

Data are released under the Open Government Licence, which encourages the re-use of our statistics as long as certain conditions are observed.