PROMs Methodologies

All four procedures - groin hernia, knee replacement, hip replacement and varicose vein - have scores for the EQ-5D™ Index and EQ VAS. Hip replacement, knee replacement and varicose vein procedures each have their own condition-specific measure, which combine into a single score a patient's answers to a number of health questions of particular relevance to their procedure.

EQ‑5D™ Index

The EQ-5D™ Index collates responses given in 5 broad areas (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression) and combines them into a single value. The EQ-5D™ Index was developed by the EuroQol Group. EQ-5D™ is a trademark of the EuroQol Group.

EQ Visual Analogue Scale (EQ VAS)

EQ VAS is a simple and easily understood 'thermometer'-style measure based on a patient's self-scored general health on the day that they completed their questionnaire, but which provides an indication of their health that is not necessarily associated with the condition for which they underwent surgery and which may have been influenced by factors other than healthcare. The EQ VAS was developed by the EuroQol Group.

Oxford Hip Score (OHS)

The OHS contains 12 questions on activities of daily living that assess function and residual pain in patients undergoing total hip replacement surgery. The OHS was designed, developed and validated by workers within public health and at the University of Oxford.

Oxford Knee Score (OKS)

The OKS contains 12 questions on activities of daily living that assess function and residual pain in patients undergoing total knee replacement surgery. The OKS was designed, developed and validated by workers within public health and at the University of Oxford.

Aberdeen Varicose Vein Questionnaire (AVVQ)

The AVVQ allows patients to self-assess the severity of their varicose veins via a 13-item measure covering all aspects of their varicose veins including physical symptoms such as pain, ankle oedema, ulcers, the effect on daily activities, and cosmetic issues.

Use of the AVVQ was discontinued in October 2017 with the removal of the collection of varicose vein questionnaires as part of the response to the NHS England PROMs consultation*

Sample questionnaires                                         

Both Hospital Episode Statistics (HES) and PROMs data are updated on a cumulative basis. Providers are able to resubmit data for any previous period within the current financial year where that data has been refreshed since the last submission. For example, data relating to an episode that took place in April submitted by a provider in June with incomplete clinical coding may be resubmitted in July once coding is complete.

The methodology that matches a HES episode to a pre-operative questionnaire uses a four-stage process that looks at a combination of patient identifiable fields, provider codes, operation codes and dates. It uses a 'match ranking' system in which a score is attributed to each part of the linking process, where the quality of the match is denoted by the rank, with the lowest rank (i.e. 1) being the highest quality match. The scores for each possible match are compared and the highest match is chosen.

The four stages are:

1. Patient matching
2. Provider matching
3. Date matching
4. A 'tie break' to pick the best match if an episode matches to more than one pre-operative questionnaire by combining the scores from the three stages.

This approach has the advantages of maximising the matching rate by attempting several different matches on the same data and allowing the easy monitoring of quality and confidence of the match. 

A guide to PROMs Methodology

PROMs scores are adjusted using statistical models which account for the fact that each provider organisation deals with patients with different case-mixes. This allows for fair comparison between providers and England as a whole. Random variation in patients means that small differences in averages, even when case-mix adjusted, may not be statistically significant. We therefore calculate 'control limits' which represent boundaries, providers falling outside of which may be stated with statistical validity to be significantly better (if above the upper limit) or significantly worse (if below the lower limit) than England as a whole. These significantly better or worse providers are known as outliers.

The case-mix adjustment models were initially developed by the Department of Health and subsequently revised by NHS England. The models are periodically reviewed to ensure that the methodology remains sound over time and are reviewed and approved by the Indicator Assurance Service (IAS) before implementation. Currently, groin hernia and varicose vein procedures use second generation Casemix adjustment methodology, whereas hip and knee replacements are on the 3^rd generation, more recently updated to account the differences primary procedures and revision procedures. In-depth details of the methodologies are hosted on the NHS England's PROMs page with links below:

• Patient Reported Outcome Measures (PROMs) - An alternative aggregation methodology for case-mix adjustment
• Patient Reported Outcome Measures (PROMs) in England: Update to reporting and case-mix adjusting hip and knee procedure data
• PROMs risk adjustment methodology guide for general surgery and orthopaedic procedures
• Annexe 1: Coefficients for primary hip replacement models
• Annexe 2: Coefficients for hip replacement revision models
• Annexe 3: Coefficients for primary knee replacement models
• Annexe 4: Coefficients for knee replacement revision models

Modelled records are the number records where both the pre- and post-operative questionnaires have been completed, the questionnaire pair has been successfully linked to a record of hospital inpatient activity and key data items used in the case-mix adjustment methodology have valid values recorded. Average case-mix adjusted scores are calculated only where there are at least 30 modelled records, as the statistical models break down with fewer records and aggregate calculations on small numbers may return unrepresentative results

For PROMS data prior to 2015/16, where necessary for the protection of patient confidentiality, figures between 1 and 5 have been suppressed and replaced with an asterisk ('*') and derived figures have also been suppressed. Further suppression has been applied where it would otherwise have been possible to identify a suppressed figure by derivation from other published figures. The 2014-15 PROMs final publication (published August 2016) is the last publication to use this methodology.

For 2015/16 PROMs data and beyond, a new methodology has been applied, removing secondary suppression with figures between 1 and 5 have been suppressed and replaced with an asterisk ('*') where the parent figures is also between 1 and 5, for example if there are 6 pre-operative questionnaires completed, and only 3 post-operative questionnaires, then the 3 will not be suppressed. This change to the suppression methodology was announced November 2015. The new methodology will not be applied to previous years.

For further information, please see Methodology change to suppression rules or email [email protected].