In response to the Francis Enquiry into Mid Staffordshire Hospital, we were asked by the Quality Improvement Committee (a sub-group of the National Quality Board) to establish an assurance process for handling work associated with the development of indicators.
An indicator is something which provides evidence that a certain condition exists, or certain results have or have not been achieved. Indicators can measure inputs, processes, outputs and outcomes. This is different to a metric, which is a measure of a known attribute.
Some of the attributes of an indicator include:
- clear statements about the intended goal or objective
- descriptions of how a measure is expected to be used to judge quality
- any thresholds or standards which are expected to be applied
- whether it is expected to be used in combination with other indicators
- alerts to an issue that needs further investigation
The number of emergency readmissions following an appendectomy is a metric. The rate of readmissions is an indicator.
Benefits of the assurance
The principal benefit of assurance is that it instils transparency. By accessing indicators based on transparent methodologies, users can be clear on what is being measured and how. Users can also identify any limitations associated with the approach.
Transparency allows scrutiny of methodologies, which drives improvements in quality. This means decisions can be made based on the best information, avoiding costly and unnecessary investigations.
Assurance also provides users with confidence in the indicator and benefits producers by allowing them to demonstrate that they can be considered a trusted source of information.
The aims and benefits of the IMAS are to:
- provide a transparent record of indicators and how they are derived
- ensure robust statistical methodologies are used in indicator development
- assure that indicators answer the question being posed and are suitable for the intended audience
- reduce indicator duplication and discourage the development of multiple indicators providing the same information, reducing confusion amongst users
- provide assurance to the users that the indicator is not likely to change over time without due consideration
- gain recognition for an indicator through registering it in the library
The assurance process
To begin the assurance process, you need to complete and submit an application form.
The application form consists of five sections describing the indicator:
Introduction / overview
Title of the indicator, who is the applicant, who is the sponsor, what set or domain it belongs to.
Rationale and policy basis
Why is this indicator needed, what are the reasons and evidence for developing this indicator, do any government strategies or policies support this indicator.
Presentation / interpretation
Is the indicator clearly presented for users to understand, what does ‘good’ / ‘bad’ look like, targets or benchmarking.
What is the data source, how will it be collected, data fields, data filters, data sets.
Construction and risks
How will the indicator measure be calculated, the numerator and denominator. Confidence intervals. Are any limitation, risks or perverse incentives associated with the indicator explicitly stated.
Upon receipt of the application, the IMAS team will conduct an initial review of the form for completeness, quality and information. All findings will then be discussed with the applicant to ensure that the submission is complete and robust enough to satisfy the Methodology Review Group (MRG) that the indicator is fit for purpose.
Once the findings are amended and agreed, the application will go to the MRG for assessment. The applicant will attend this meeting to answer any queries or questions.
The Methodology Review Group
The Methodology Review Group (MRG) is made up of statistical, methodological and data experts from arms-length bodies, such as:
- NHS Digital
- Public Health England
- NHS Improvement
- NHS England
- members from local councils and academia
In addition, MRG also consults with independent experts for their knowledge relevant to the specific field of an indicator. These can include:
MRG determines whether the indicator is fit for purpose, they provide an assessment of how well the indicator meets the assessment criteria and whether the indicator is of sufficient quality and transparency. Advice and recommendations from MRG are fed back to the applicant, with the intention of strengthening the application.
Once the application has been amended by the applicant, it is re-presented to the MRG. The MRG then recommend an indicator rating and a period for the indicator to be assured, which can be up to a maximum of five years. The indicator will then be presented to the Indicator Governance Board (IGB) for final approval.
The Indicator Governance Board
The Indicator Governance Board (IGB) membership is drawn from a broad range of organisations, with decisions being made on the basis of collaboration and shared sovereignty.
The IGB discuss governance issues, such as:
- duplication of indicators
- general strategic value of new indicators
- impacts across organisations
- issues relating to information governance and interpretation
If the IGB have any queries about the application, they will be fed back to the applicant via the IMAS team.
If the IGB approve the indicator as being fit for purpose, robust and of high quality, it will be uploaded to the National Library of Quality Assured Indicators. The IGB also sets the length of the accreditation period, which can be for a maximum period of five years.
The outcome of IGB deliberations will be communicated to the applicant and sponsoring body.
Six months prior to the lapsing of its accreditation, IMAS will contact the sponsoring body to determine whether the indicator should be renewed or retired.
The National Library of Quality Assured Indicators
Regulations underpinning the Health and Social Care Act 2012 called for NHS Digital to establish a database of quality indicators in relation to the provision of health services and social care.
The National Library of Quality Assured Indicators consists of assured quality indicators that have been assessed as suitable for inclusion.
In the library you will find all the assured indicators and information about them, including:
- the purpose, definition and methodology of the indicator
- the application form, appraisal log, technical specification and quality statement (in the resources section)
The European Standard Population (ESP) approach allows for comparability across the Health and Social Sector and provides a universal standard for direct standardisation.
Indicator Governance Board - Chair's report
If you have any additional questions regarding IMAS, MRG or IGB, please contact IMAS by emailing firstname.lastname@example.org or by calling 0113 241 6100.