Multiple Submission Window Model for MHSDS

Since April 2020, NHS Digital has implemented a Multiple Submission Window Model (MSWM) for MHSDS, which allows providers to retrospectively submit data for a specific reporting period once the initial provisional and performance submission windows have closed.

This webpage is particularly aimed at data providers so they understand how data should be submitted and how the data is used. This webpage was updated in July 2021 to present the latest position.

In general, the Multiple Submission Window Model should only be employed where:

• a new provider starts submitting data mid-year for the first time
• an issue was identified in a provider’s submission for that month that requires resubmission
• the resubmission of a particular month’s data has resulted in data associated with earlier or later months being impacted

The resubmission of data is optional, and we anticipate providers will use this process only in exceptional circumstances. There is no need to resubmit data unless the resubmitted data are considered to be of higher quality than the original submission or if the data being resubmitted has a knock-on effect on other data (for example, where a change to April’s submission impacts the May and June submissions, then all three months should be resubmitted).

The provisional window and performance windows will remain (previously the primary and refresh windows). Data can then be resubmitted in each update window. 

This financial year (2021-22) the MHSDS will undergo a version change to MHSDS Version 5, halfway through the year impacting the MSWM resubmission process. This year there will be two final windows to resubmit data one for each version of MHSDS, MHSDS v4.1 and MHSDS v5.0.  The first will be in November (for April to September 2021 data relating to MHSDS Version 4.1) and the second in May 2022 (for October 2021 to March 2022 data relating to MHSDS Version 5.0). The ‘last good file’ submitted for each month will be used to generate final statistics at the end of the year.

The MSWM only allows resubmission of data for months in the current financial year and MHSDS version. For example, once the provisional and performance submission windows have closed for May 2021 it will be possible to make a resubmission for this month’s data up until 18 November 2021 (the end of the submission period for v4.1 2021 data).  After this date, no updates using either v4.1 or v5 can be made for April to September 2021 data.

These cut offs will apply to all data associated with that part of the financial year, thus the resubmission window will in effect be shorter than usual for the period April 2021 to September 2021. As before, as the window progresses the time allowed for resubmission also reduces. For example, January 2022 data can be resubmitted until May 2022 , while March 2022 will have no resubmission period (just the standard provisional and performance windows, which end in May 2022).

To allow NHS Digital to process the new data effectively there have been several changes to how data is submitted:

• as per now, each month of data must be submitted in a separate file
• the SDCS Cloud submission portal will allow the month being submitted to be specified by the user
• there has been a change to the validation applied in the MHS000 Header table
• data item ORGANISATION IDENTIFIER (CODE OF PROVIDER) will now be rejected if the code provided does not match a code that is live in the national organisation identifier code tables for the reporting period - this is designed to improve data quality, by preventing data from flowing that is attributed to an organisation that does not exist at the specified point in time
• where your organisation has changed during the financial year you will need to submit data prior to the organisation change using your log in details for the old organisation
• there has been a change to the derivation FILE TYPE found in the MHS000 Header table which now indicates how many months after the reporting period the file was submitted for example, May data submitted in August would show as ‘3’.  With the change in version the FILE TYPE count will reset to 1 when the migration to version 5 activity is enacted
• the change makes it clear how long after the reporting period the data was submitted, and can therefore also be used to identify during which window the submission was made - for example, an extract that relates to data collected during April, and has the code ‘6’, would have been submitted in the October submission window.

Until March 2020 the Mental Health Services Data Set (MHSDS) was operated on a monthly submission cycle, where data providers had two opportunities to submit data – the provisional submission which opened one day after the end of the reporting month, and the refresh submission, which opened one month after the reporting month.

These two submission windows were the only opportunity that providers had to submit data for that specific reporting period. If data was found to be incorrect or missing, it resulted in an inaccurate reflection of mental health services when viewing data and analytical reports. This inflexibility resulted in additional data collection burden on healthcare providers.

It was therefore agreed that the Multiple Submission Window Model described in Information Standard DCB0011 Amd 43/2019 would be implemented in advance of the implementation date of MHSDS v4.1 i.e. within the financial year 2019-2020.

Providers should:

• submit the best possible data at least once in the Provisional / Performance Window
• only resubmit data where there is a clear need to. For example, when a new provider starts submitting data mid-year for the first time or an issue was identified in a provider’s submission for that month that requires resubmission
• only resubmit the data they need to resubmit, not the entire financial year each time, unless there are knock on implications and this is necessary. For example, the resubmission of a particular month’s data has resulted in data associated with earlier or later months being impacted

NHS Digital and NHS England will be regularly monitoring the number of resubmissions of providers. NHS Digital will be highlighting misuse of the MSWM and any issues to NHS England regional colleagues, and commissioners, as part of the quarterly Deep Dives.

The way the data is employed will change slightly for those measures that consider performance of the provider over time, taking advantage of the MSWM. This is described below:

The provisional report will continue to be produced as it is important to have an early view of the data especially for issues such as Delayed Transfers of Care (DToC).

The performance report (previously the final report) will be produced using the same timeframe that is currently adopted, i.e. following the Performance and Provisional submission windows.

These reports will be classed as experimental statistics, containing all measures that would have been included previously and any new measures will be added to this publication in due course.

The provisional and performance reports will employ all relevant data available to NHS Digital at the time the reports are issued (up to the closure of the performance submission window). This takes advantage of the MSWM as follows:

• if a provider has resubmitted historic data (resolving issues) or submitted historic data for the first time (catch up), this will be included in the calculation of measures that consider data over a period of time thus better reflecting the performance of the provider, an example measure that would be improved is MHS69 - the number of children and young people receiving at least two contacts
• prior to the MSWM, the measure would have been calculated using incorrect data or missing data if it had been submitted in error or not been submitted during the provisional or performance window
• measures that reflect the performance of the provider during the provisional or performance window will continue to be produced as before, using only data submitted in the provisional or performance window

Any data submitted post the creation of the provisional or performance reports will not be included in the calculation of measures for these reporting months until the final report, see below:

The notes and data quality documentation associated with these publications will include information relating to the MSWM, and the impact it may have had on the publication (at a high level).

On a monthly basis the MSWM tracker shall be published, this will record the new submissions that have been made for that month post the creation of the performance (previously the final) publication. For instance, if a provider had not submitted data by the time of the publication and now has or a provider has increased or decreased the volume of records for that month substantially, this will be stated. This will not result in an update to the performance publication.

In line with the Annual Bulletin or earlier; once all the submissions have been finalised, that is, post May of the following financial year, a final publication report shall be created for each month. It is not proposed that the changes/impact from the Performance Publication are assessed, as this will have been documented through the MSWM Tracker described above. This new final report shall be added to the publication page for the relevant month.

NHS Digital will continue using data submitted during the first two submission months (Provisional and Performance data) to calculate the Data Quality Maturity Index.

NHS England and Improvement will use data published by NHS Digital, where possible, to monitor performance against national standards. Usage of the MSWM will also be closely monitored and fed back to the system.

Different approaches may be used in other NHS England and Improvement products, but this will be clearly indicated.

MHSDS data will continue to be used in the current range of products and many of these will use the updated data available during the year.

NHS Digital extracts are built from data generated at window-close snapshot time. Extracts will reflect the most recently submitted version of data and will not be available for earlier data submissions that have been replaced.

Therefore, the data included in the extracts will be the latest data as at the last end of a specific reporting period point, as set out in the submission timetable.

This means that analyses undertaken for any annual audits (e.g. Early Intervention in Psychosis – National Clinical Audit of Psychosis) will be using any updated data submitted by providers as part of the Multiple Submission Model Window. This also applies to data used for the NHS Oversight Framework, and used by the Care Quality Commission, Public Health England and NHS Improvement.

This change follows consultation with a range of stakeholders and the publication of the Information Standard Notice (ISN) for the Mental Health Services Data Set (MHSDS) v4.1 (commenced April 2020).

Multiple Submission Window Model (MSWM) Review

NHS Digital conducted a review of the use of the mental health MSWM at the end of the 2020/21 financial year. The data showed that there was no significant difference in the measures presented when either the MSWM or only the performance-provisional submissions were used. Half of all providers did make use of the MSWM to make changes to their data, whether that was retrospectively changing data or submitting data after the deadline. It was also noted that the MSWM tended to see most usage towards the end of the financial year, with the number of providers utilising the model across the year increasing substantially. This pattern is also seen in much of the data where metrics calculated using the MSWM or the performance-provisional submissions show very little change across the year until the end of the financial year where the impact of those resubmissions is seen most.

If you have any feedback please email our Contact Centre on [email protected] or telephone us on 0300 303 5678.