NICE Technology Appraisals in the NHS in England (Innovation Scorecard) To March 2020

The Innovation Scorecard is an action taken from The Department of Health’s (DH) paper ‘Innovation, Health and Wealth’ (Dec 2011).

Further information and report can be found at: https://www.gov.uk/government/news/accelerating-adoption-of-innovation-in-the-nhs

More recently, the Accelerated Access Review (AAR) makes a number of references to the development of the Scorecard in particular its role in measuring the uptake of medicines and potential to measure other technologies and in future, the Innovation Scorecard should be the single source of information on the use of innovation in the NHS.

Further details and final report can be found at: https://www.gov.uk/government/organisations/accelerated-access-review

The Innovation Scorecard aims to support and drive NHS compliance with NICE Technology Appraisals (TAs) and was designed to help the NHS identify variation which can then be justified, challenged or acted upon.

It is published by the NHS Digital on behalf of the Office for Life Sciences. The work is informed by collaborative working with colleagues from: the Association of the British Healthcare Industries (ABHI), Cabinet Office, DH, NHS Digital, the NHS, NHS England, NICE, Office for Life Sciences, and the pharmaceutical industry. Recent developments include:

• inclusion of medicine groupings
• re-introduction of medical technologies
• development of a 1-page key points infographic
• enhanced presentation of the data in a web platform tool
• incorporating work from the ‘Use of NICE Appraised Medicines in the NHS in England’ 

Technology appraisals  are recommendations on the use of new and existing medicines and treatments within the NHS, including medicines, medical devices, diagnostic techniques, surgical procedures and health promotion activities.

Some medicines and treatments may be covered by more than one technology appraisal. Each technology appraisal may contain more than one recommendation. NICE classify their recommendations into four categories:

• Recommended - the medicine or treatment is recommended for use:
  • In line with the marketing authorisation from the European Medicines Agency (EMA) or from the Medicines and Healthcare Products Regulatory Agency (MHRA) or
  • In line with how it is used in clinical practice in the NHS
  • or both
• Optimised - the recommendations have a material effect on the use of a medicine or treatment, and it is recommended for a smaller subset of patients than originally stated by the marketing authorisation. This test of materiality takes into account advice from clinical experts on the anticipated use of the technology in routine clinical practice. In some instances, an optimised recommendation is made because the committee considers that a medicine or technology is only a cost-effective treatment option for a specific group of people; for example in people who are resistant to or cannot tolerate other medicines.
• Only in research - The medicine or treatment is recommended for use only in the context of a research study, for example, a clinical trial. Often, particularly in the case of promising new technologies, sufficient clinical evidence has not been collected at the time of the appraisal and so the Appraisal Committee is unable to recommend the technology for use in the NHS until further evidence on its effectiveness is available for re-appraisal.
• Not recommended - the medicine or treatment is not recommended. In most instances, a technology will not be recommended if there is a lack of evidence for its clinical effectiveness or if the technology is not considered to be a cost-effective use of NHS resources, compared with current NHS practice.

The technologies included in an appraisal may not be the only treatment for the condition recommended in NICE guidance, or otherwise available in the NHS. Therefore, if a NICE technology appraisal recommends use of a technology, it is as an option for the treatment of a disease or condition. This means that the technology should be available for a patient who meets the clinical criteria set out in the guidance, subject to the clinical judgement of the treating clinician. The NHS must provide funding and resources when the clinician concludes and the patient agrees that the recommended technology is the most appropriate to use, based on a discussion of all available treatments.

Some medicines and medical technologies previously covered by TAs are now included in clinical guidelines or in best practice guidelines.

NICE Technology Appraisals for Medicines

Further guidance on NICE TAs and those technologies that are considered by NICE guidance other than the TA process is available at: http://www.nice.org.uk/About/What-we-do/Our-Programmes/NICE-guidance/NICE-technology-appraisal-guidance

The Scorecard includes a range of data from a number of sources which includes:

• Prescription data from NHS Prescription Services, part of the NHS Business Services Authority
• Hospital Pharmacy Audit data from QuintilesIMS
• Pharmex data from Commercial Medicines Unit (CMU) at NHS England
• Hospital Episode Statistics (HES) data from NHS Digital
• Mid-year population estimates from Office for National Statistics (ONS)
• Defined Daily Doses (DDD) from World Health Organisation (WHO) Collaborating Centre for Drug Statistics Methodology
• Actual Daily Doses (ADD) from NHS Digital
• Sales data from NHS Supply Chain

The Innovation Scorecard is published by NHS Digital on a quarterly basis with the first release in January 2013.

All releases of the Innovation Scorecard are available on the NHS Digital website and can be accessed at: https://digital.nhs.uk/data-and-information/publications/statistical/nice-technology-appraisals-in-the-nhs-in-england-innovation-scorecard

A process for selecting the medicines and medical technologies for inclusion was developed to ensure a systematic approach for inclusion within the scorecard. The Strategic Metrics Group and Technical Working Groups agreed a series of inclusion and exclusion criteria, and a rationale for any medicines excluded.

Inclusion criteria for medicines

• The medicine has a TA published from 2013 onwards, which is positively appraised by NICE. The date will move forward a year as part of an annual cycle starting with the October publication.
• TAs published prior to 2013, which are deemed to be innovative will be considered for inclusion.
• The medicine has a TA prior to 2013 but is of interest to the NHS. This may include those medicines which do not have widespread use across the whole service, for example, where there is unwarranted variation or where there is known significant therapeutic value.
• Following publication of the TA, a period of six months must be allowed for data availability from QuintilesIMS.

Exclusion criteria for medicines

• Where a significant proportion of data is not currently available, for example, those medicines which can be purchased over the counter, or dispensed via routes other than hospital or community pharmacies, etc.
• Where the medicine is not new and therefore, not considered to be innovative as the treatment would be well established already.
• The patient population is small and therefore there is a risk of identification when presenting the data
• Where inclusion of the data available is misleading or of limited value to the NHS

Inclusion criteria for medical devices and diagnostics

• Any product receiving a positive NICE Technology Appraisal, or medtech guidance assessment or diagnostic products assessment (and meeting criterion ii) should automatically be considered for inclusion on the Innovation Scorecard.
• Products subject to guidance less than 10 years old is the indicative timescale for including medtech and diagnostic products on the Innovation Scorecard, but this is a guide not a rule.
• Heavier weighting should be given to products that are aligned with key priorities e.g. the NHS Innovation Accelerator, the Innovation Tariff or the AAR. Clinical or system priorities (e.g. as stated in the 5 Year Forward View) should be similarly weighted.
• Any product being included on the Innovation Scorecard should have a clear usage/uptake data source(s).

Inclusion criteria for the estimates report

The inclusion of estimates to show expected uptake versus actual uptake were first included in the scorecard in the September 2014 publication. The estimate approach requires its own criteria to determine which medicines should be selected for that methodology. The “Use of NICE appraised medicines in the NHS in England” report provides historical selection criteria for those medicines included in the estimate approach.

Yes, some details are provided on the web platform tool when selecting a medicine, high level condition, medicine grouping or medical techonology. A more detailed description is provided in the Accuracy and Reliability section of the Background Quality Report.

The Prescription Cost Analysis (PCA) data published by NHS Digital covers all community dispensing in England and will not necessarily match the data used here.

Hospital trust level data in the Innovation Scorecard will not match data presented in the report on Prescribing Costs in Hospitals and the Community (published annually) as the latter is based on the costs and not the utilisation of medicines.

The Innovation, Health and Wealth Strategy included a recommendation that all NHS organisations should publish information which sets out which NICE technology appraisals are included in their local formularies by 1 April 2013.

Formularies are available on individual Trust websites. Please contact the hospitals directly with any queries relating to the formularies.

Compliance is subject to interpretation, and the measurement of compliance is not simple. An important aspect of this is consideration of the circumstances under which a medicine is administered or a medical technology used, and whether this aligns with the precise recommendations within the NICE guidance. The ”Innovation, Health and Wealth” report sets out actions for the NICE compliance regime, so readers are advised to view this report.

The data presented here is insufficient to determine if an organisation is or is not compliant with NICE guidance.

The Innovation Scorecard should not be used for performance management. It is intended to identify where variation in the adoption of TAs may exist between healthcare organisations and for these organisations to understand, be challenged and explain any variation. This is based on the assumption that reduced variation will result in improved quality of care.

It is important to be aware that observed use of a medicine or technology may differ for a range of reasons and should not be assumed to definitely indicate either ‘under’ or ‘over’ prescribing or implementation. It is recommended that any exploration of the variation uses a structure such as the pyramid of investigation for special cause variation^[1] (see diagram below).

More likely explanations are listed towards the bottom of the pyramid, and so it is suggested that these are investigated first. The data presented in this publication requires careful interpretation and should be used in conjunction with other indicators and information from other sources that together form a holistic view of use of medicines and medical technologies. For example, a technology may not be the only treatment for a particular condition recommended in NICE guidance, or otherwise available in the NHS. Medicines are generally recommended as options for treatments. Other options may include non-appraised medicines or other appraised medicines. Therefore variation in the use of individual medicines would be expected.

Assessment of compliance cannot be made using currently available data.

Indicators of variation may be more appropriate for CCGs than for Trusts, as medicine use in primary care is more likely to be for the management of common long term conditions. Some appraised medicines are used wholly or mainly in tertiary care hospitals; others are more likely to be used in less specialist hospitals. A medicine which is used for a very rare condition (tens of doses per CCGs or Trust per quarter) would normally be expected to show wider variation between organisations.

Variation in the use of medicines or medical technologies between NHS organisations may be due to a number of valid factors including:

• Natural variation in populations, both in demographic profile and disease prevalence
• Variation in presentation to the NHS by the relevant populations
• Variation in choice of preferred treatment option at the local level
• Variation in the use of alternative products or procedures
• Differences in the extent to which local utilisation information is available
• Differences in services provided between organisations, for example differences in the extent to which a service is provided by primary or secondary care
• Difference in levels of informed patient dissent to intervention

A detailed examination of the reasons for variation for individual technologies is beyond the scope of this report.

The publications show use of medicines and technologies over time. Data on the Innovation Scorecard is presented by calendar quarter which covers up to 5 years of data at the National (England) level and up to 2 years of data at lower NHS Organisation levels. 

It is not appropriate to draw conclusions from direct comparisons of the data in each of the scorecards, as many factors may be responsible for any apparent variation. These include: 

Clinical and demographic issues:

• differences in populations served, local prevalence of disease, services provided and changes in clinical practice.

Data issues:

• The NHS reorganisation has led to changes in the individual hospitals included in some Hospital Trusts, and there have been changes in the general practices included in some CCGs.
• Data which has been standardised by FCE days of hospital care (introduced in March 2014 publication) cannot be compared with data standardised by FCE bed days in previous publications.

Data in the Scorecard are presented in several ways – based on available data at specific NHS organisation levels.

Data for medicine utilisation is reported at National (England) and NHS Region level. Medicine use in primary care is also shown at CCG and STP level, and medicines purchased in secondary care are shown at hospital trust level.

Medical technology data is shown at NHS Region in which the patient is resident.

The web platform tool allows comparison of medicine use at CCG level and medicine purchased at NHS Trust level. Additionally, maps presented on the tool shows use based on quintile proportions. Data for all organisations are provided in the CSV files.

Data is provided at NHS Hospital Trust level. The majority of trusts are made up of a number of local hospitals.

No, we are legally obliged to avoid disclosure of potential identification of individual patients by implementing suppression handling where required.

NHS organisations differ widely in the populations they serve and so data which does not take this into account can be misleading.

For CCG, Region and national data the number of resident patients has been used to take account of the size of population served. The smallest CCG has a population less than 65,000. Larger CCGs have populations of over 850,000.

For hospital Trusts data the number of FCE days of hospital care for the time period under consideration (taken from the Hospital Episode Statistics data) has been used to standardise the data. The values vary significantly, with more specialist hospitals, for example, the Royal National Hospital for Rheumatic Diseases NHS Foundation Trust reporting fewer than 10,000 days of hospital care per year, whereas larger Trusts such as Bart’s Health NHS Trust report over 800,000 days of hospital care per year.

Trust level data should not be compared with the national, Region or CCG data.

This scorecard covers some highly specialised medicines and technologies so differences in use across organisations are to be expected.

Medicines presented as grouped medicines are also standardised either by resident populations, FCE days of hosiptal care or by national registers i.e. QOF. Further details can be found in the grouped medicines methodology specification document.

Data on the number of patients diagnosed with a particular condition, eligible for a specific intervention, or receiving an intervention may be recorded locally but are not reported centrally by the NHS; therefore the eligible population for any TA can be problematic to estimate. To develop estimates of the eligible population, information is required to refine population numbers to the particular circumstances where the medicine is recommended by NICE; from overall disease prevalence to the proportion of patients within a particular stage of a disease and then to the particular indication recommended by NICE. In some cases further details are also required, for example, the proportion of patients likely to discontinue treatment or choose alternative treatments.

The NHS currently does not have any robust data reporting or collection process which gathers data which can be used to measure compliance with NICE TAs.

The data sets used in this publication were developed for other specific purposes and are shown here to provide an indication of the implementation of NICE appraised technologies. Therefore users of the Scorecard should be familiar with the limitations of the data when used for this purpose.

Patients can receive medicines from the NHS by a variety of routes. The most common is to receive a prescription from their general practitioner or other community based prescriber, which is then dispensed by a community pharmacy. Hospitals also administer and supply medicines to patients.

However there are a number of other ways patients can receive medicines which may limit the coverage of centrally available data, and result in under-reporting of utilisation. Some of the main areas are described below:

• Medicines are supplied directly to a patient without a prescription being issued or the supply being recorded in a system which supports central collection of data. For example:
  • Some medicines (e.g. for smoking cessation or contraception) are supplied directly to patients from specific clinics, and this use does not appear in any central data collection.
  • Some mental health services or clinics purchase medicines and supply or administer them directly to patients without using a prescription or order in such a way that relevant information is not routinely captured in national data sets.
  • Some urgent supplies of medicines are supplied directly to patients and this use does not appear in any central data collection, for example, supplies provided by ‘out of hours’ services.
• There are some known gaps in the central data collections, including:
  • Homecare: Supplies of medicines are delivered directly to the patient’s home, and may not be routinely recorded on hospital pharmacy systems. This service is generally contracted to a commercial supplier by hospital Trusts. The homecare market is a significant proportion of total medicines use in secondary care.
  • Some medicines are provided by an aseptic unit where the medicine has been individually prepared for administration to a particular patient. This data may not routinely appear in the hospital pharmacy systems.
  • Outsourced dispensing: Hospitals are increasingly using local commercial suppliers to dispense medicines to patients attending out-patient clinics and these supplies may not be routinely included in hospital pharmacy systems.
  • Hospitals can contract with the private sector to provide specific care or administer specific medicines, which are not recorded in the data from the commissioning hospital. This is most commonly seen with particular medicines, such as ranibizumab, where specific monitoring is not required.

Some medicines are included in multiple TAs; and some medicines are licensed for other indications than those covered by TAs. The available datasets do not allow identification of utilisation by indication, therefore it is not possible to report use for a particular condition.

The figures are derived from both the QuintilesIMS Hospital Pharmacy Audit Index (HPAI) database and the databases maintained by the Prescription Services Division of the Business Services Authority (NHSBSA). This data is reported by Defined Daily Dose (DDDs) where a DDD value for the medicine is available from the World Health Organisation (WHO), otherwise by mgs, tablets, units, vials or implants as appropriate.  Note that DDDs are generally not available for chemotherapy medicines. A new measure was introduced in January 2016 publication for medicines an Actual Daily Dose (ADD).

The data is standardised by 100,000 resident population.

Further information on the national cost of NICE appraised medicines can be found in the following NHS Digital publication:

• Prescribing Costs in Hospitals and the Community (previously Hospital Prescribing)

Further information on national medicines utilisation in primary care in England is available in the following NHS Digital publication:

• Prescriptions Dispensed in the Community

Quarterly utilisation data and charts for each medicine are provided from the latest 5 years at national level, and shown by total and by three provider sectors:

• Primary care: Prescribed and dispensed in the community – FP10 prescriptions
• Prescribed in the hospital but dispensed in the community – previously known as FP10HP prescriptions
• Secondary care: Supplies recorded in hospital pharmacy departments

On 1^st April 2015 the NHS Area Teams were reduced by NHS England from 27 to 13 Regions. Further information, including mapping files, has been published by the Organisation Data Service (ODS) on the NHS Digital website: https://digital.nhs.uk/services/organisation-data-service

Quarterly utilisation data and charts for each medicine are provided for the latest 2 years at NHS Region level, and shown by total and by three provider sectors:

• Primary care: Prescribed and dispensed in the community – FP10 prescriptions
• Prescribed in the hospital but dispensed in the community – previously known as FP10HP prescriptions
• Secondary care: Supplies recorded in hospital pharmacy departments

To help users to identify variation and interpret the data, quintiles have been used to assign Regions into five groups, for each quarter.

The data is derived from several sources and uses a range of measures:

• Volumes of NICE appraised medicines used in primary care taken from prescribing data produced by the NHSBSA.
• Physical quantities of purchases of NICE appraised medicines by NHS hospital Trusts using purchase data provided by the Commercial Medicines Unit (CMU) at NHS England.
• Physical quantities of NICE appraised medicines sold to NHS hospital Trusts using sales data provided by pharmaceutical companies.

These data cannot be added together.

Utilisation data for use of NICE appraised medicines in primary care has been extracted for the latest eight quarters. This data is presented in DDDs per 100,000 resident population, where available. These figures are not related to any measure of need (other than the total population). This publication displays data for the current CCG list and has mapped data from closed CCGs to their successor organisations.

Note: Since many of the medicines are used only in secondary care, it is expected that some CCGs will have a zero value.

Medicines utilisation data by hospital Trust is not centrally collected; however data relating to some purchase of medicines by hospital Trusts is available. This cannot be regarded as providing a measure of current utilisation, as there are known gaps in the data collected and some Trusts may purchase on behalf of other Trusts.

There are changes to the NHS hospital Trusts included in each publication, generally due to Trust mergers. A hospital Trust may cover several separate hospital sites.

In this publication only acute care Trusts are included in the Innovation Scorecard. All mental health trusts and community hospitals are excluded. Where a Trust offers a combination of services, which requires them to submit acute care data, they will be included. This does not impact on the National, NHS Region or CCG figures for these medicines.

Purchase and sales data and charts are provided for the latest eight quarters. Data may not be available for all Trusts. The information reported reflects the structure of a particular Trust as per the latest Trust list.

Hospital Trusts vary significantly in the size of the populations they treat and the services they provide. For example some treat just children and others provide services related only to cancer or other specific conditions. The catchment population for a hospital is influenced by the condition being treated, so local population figures cannot be used for standardisation.

This data is presented in mgs or units per 100,000 Finished Consultant Episode (FCE) days of hospital care in total (and not just for a particular therapy area). FCE days of hospital care include FCE bed days and FCE day cases.

These figures are not related to any measure of need (other than the total number of days of hospital care).

Purchase data is supplied by NHS Hospital Trusts to the Commercial Medicines Unit at NHS England. Previously Trusts were invited to provide comments on their data. However due to the burden to Trusts, this activity has ceased. Many Trusts have reported that purchase data is not appropriate for the assessment of utilisation or compliance and could be misleading as presented here. However, it does show variation in purchase patterns which can be explored further. There are known gaps in the data, therefore the figures presented are likely to be an underestimate of total purchases or supply of medicines into Trusts.

Pharmaceutical companies were invited to provide data for sales of medicines to hospital Trusts. These are contextual data, included in the publication to aid interpretation of the purchase data that is supplied for all individual medicines. It is hoped that industry have more complete information on supplies to the NHS, and be able to fill some of the known gaps in the data available centrally to the NHS. Although a small number of companies have provided data for previous publications, currently, there is no company supplying data. Work is on-going to improve the contribution of industry data.

In January 2016, medicine groupings were introduced to the Innovation Scorecard. The medicine grouping methodologies have been developed by analysts and pharmacists at NHS Digital, ABPI, OHE, NICE, OLS and NHS England. All methods have been reviewed and agreed by the technical working group for the Innovation Scorecard. Comments and feedback on this new section of the publication would be appreciated. Please email any feedback to enquiries@nhsdigital.nhs.uk.

These groupings have been developed to show combined use of medicines that are options for treatment of a specific condition, and where a TA covers more than one medicine for the same indication or two or more TAs cover the same specific condition. It is more informative to compare uptake of combined options for treatment than only showing uptake of the individual medicines. These groups only contain those medicines with a TA. Other options for treatment may be available but are not reported in the Innovation Scorecard.

Data for the medicine groupings is not comparable to the data published in previous Innovation Scorecard publications.

Actual Daily Dose (ADD) is a new prescribing measure developed for and introduced to the Innovation Scorecard in January 2016. They have been developed where current prescribing measures, such as DDDs, are not available or representative of prescribing practices in England.

• Unlike DDDs that use a value for a drug (chemical substance), ADDs use a unique value for each presentation based on the actual dose likely to be taken.
• ADDs assign a unique value for each presentation of a drug based on units (tablets, capsules, patches etc.) and the recommended frequency of daily use (e.g. one a day, three times a day).
• The general assumption being that each dose is equal to a unit of each preparation.
• The advantage of ADD is that a measure can be created that matches the prescribing patterns when a DDD has not been calculated; was created for a different indication (e.g. a DDD for NOACs is based on utilisation in secondary care, but different doses of medicines are used in primary care that are not covered by the DDD); the dose is variable depending on age or a dose and hence presentation is specific to an indication.
• Therefore, different presentations of the same chemical are likely to have different ADD values whereas a single DDD value for a chemical substance is usually applied to all presentations of the substance.
• For this publication, the ADDs have been created using each presentation strength of the medicines linked with licenced dose regimens and NICE guidance.

This report includes medicines where it has been possible to estimate the number of patients predicted to be treated with the medicines and comparing that to the observed use. The report is produced and written by NICE. This work is complex due to difficulties in deriving estimates of the number of patients expected to be treated and obtaining appropriate utilisation data for specific medicines and indications. Medicines may have multiple indications (some NICE appraised and some not) and there may be a number of medicines with the same indication. A TA generally recommends a medicine as a treatment option for consideration alongside other options. There are known gaps in medicine utilisation data (such as supplies made directly to patients via the homecare route or by outsourced dispensing).

For January 2016, the estimate report was reformatted to separate the estimates and measures of uptake from the more detailed methodology used to calculate the estimates. The methodology and reference data lists are now provided in a separate annex in the second half of the estimates report. Feedback on these changes made to the structure of the report and the changes to the methodology and the introduction of ranges is welcome. Any comments can be sent to enquiries@nhsdigital.nhs.uk

The Government Statistical Service (GSS) Methodological Committee recommended that estimates should include a range to provide an indication of the level of uncertainty in the data. For this publication all estimates now include a range:

Full details of the calculations are available on Appendix 1: Estimate Calculations within the estimate report.

This report was originally based on the ‘Use of NICE Appraised Medicines in the NHS in England 2012’ report published in January 2014. This publication is a result of collaborative work between NHS Digital and NICE, with contributions from the pharmaceutical industry, DH and the NHS. The report was commissioned by DH, to meet their commitments in the Pharmaceutical Price Regulation Scheme agreements (PPRS 2009 and PPRS 2014). The report can be found at: https://webarchive.nationalarchives.gov.uk/20180328130852tf_/http:/content.digital.nhs.uk/catalogue/PUB13413/

Questions regarding policy and strategy behind the Innovation Scorecard should be sent to: officeforlifesciences@bis.gsi.gov.uk

Please quote ‘Innovation Scorecard’ in correspondence.

We welcome comments and suggestions to improve this publication. Please use the associated feedback form.

Alternatively, please send them to our contact centre at enquiries@nhsdigital.nhs.uk, quoting ‘Innovation Scorecard’ in the correspondence.

Please send them to our contact centre at enquiries@nhsdigital.nhs.uk, quoting ‘Innovation Scorecard’ in the correspondence.