Maternity Services Monthly Statistics May 2019, experimental statistics

The Maternity Services Monthly Statistics reports for April 2019 onwards are the first to come from the new version of the Maternity Services Data Set (MSDSv2). 

This version of the data set has had numerous changes: to field names, table names, the clinical coding we receive and the structure of the  data set itself. Our collection portal has also been redeveloped, to enable consistency in submissions and efficiency in analysis and data linkage in the fullness of time. System suppliers are at different stages on developing their new solution and delivering that to trusts. In some cases this has limited the aspects of data that could be submitted to NHS Digital.

These changes have meant that we have not received as much data from every trust as we did in early 2019. Some tables are mandatory for upload, and therefore included in the submission as incentivised via the Maternity Incentive Scheme of the Clinical Negligence Scheme for Trusts (CNST). While all tables are mandatory, not all are required for an upload to be successful. We expect their completeness to improve over time, as they did with the previous version of the MSDS, and are looking at ways of supporting the improvements.

• All 130 trusts in scope continued to submit.
• The completeness of those submissions seems to be of lower quality. No trusts have passed our data quality tests for the clinical quality improvement metrics (e.g. whether they submitted to MSDS an adequate proportion of average monthly deliveries from the most recent HES maternity statistics).
• The submissions were timely. We extended the submission window due to technical issues, but all expected trusts submitted within the extended submission window. 

Therefore, the statistics we are presenting in this publication are reflective of the submissions we have received, but should not be interpreted as the entirety of activity that took place in the month.

This page aims to provide users with an evidence-based assessment of the quality of the statistical output of the Maternity Services Monthly Statistics publication by reporting against those of the nine European Statistical System (ESS) quality dimensions and principles appropriate to this output.

In doing so, this meets our obligation to comply with the UK Statistics Authority (UKSA) Code of Practice for Official Statistics, particularly Principle Q3, which states:

“Producers of statistics and data should explain clearly how they assure themselves that statistics and data are accurate, reliable, coherent and timely”.

For each dimension this page describes how this applies to the publication and references any measures in the accompanying monthly data quality measures report that are relevant for assessing the quality of the output.

These statistics are classified as experimental and should be used with caution. Experimental statistics are new official statistics undergoing evaluation. They are published in order to involve users and stakeholders in their development and as a means to build in quality at an early stage. More information about experimental statistics can be found on the UK Statistics Authority website.

Relevance

The degree to which the statistical product meets user needs in both coverage and content.

This publication comprises a set of reports which have been produced from NHS-funded maternity service providers’ monthly MSDS submissions. It provides the timeliest information from the MSDS.

The MSDS does not cover non-NHS funded maternity services provided by independent organisations (e.g. private clinics).

The MSDS has been developed to help achieve better outcomes of care for mothers, babies and children. Monthly reports from the MSDS include

• An HTML executive summary
• A CSV file containing the aggregate underlying data
• A CSV data quality file showing national and provider-level data quality measures - to follow
• A detailed metadata file describing all of the measures in the analysis. This includes, for each measure, how it has been constructed from providers’ submissions and how and where it is used.

Accuracy is the proximity between an estimate and the unknown true value. Reliability is the closeness of early estimates to subsequent estimated values.

Accuracy

The MSDS is a rich, person level dataset that records packages of care received by individuals in contact with NHS-funded maternity services. NHS Digital provides a number of different reports at different stages in the data flow to ensure that the submitted data reflect the services that have been provided:

For data suppliers only:

At the point of submission:

• Providers receive immediate feedback on the quality of their submission through a validation file. This file includes record-level reports of any submission errors, giving the data providers detailed information about which records produced which errors.

On receipt of processed data by NHS Digital:

• A variety of data quality checks are run as part of the validation and load process for monthly data, prior to production of this monthly release. Where there are concerns about data quality we contact providers directly so that any issues with local data extraction processes can be addressed for a future submission.

For all users:

As part of this publication, we publish a CSV file of data quality of submissions from maternity service providers.

Users of the data must make their own assessment of the quality of the data for a particular purpose, drawing on these resources.

In addition, local knowledge, or other comparative data sources, may be required to distinguish changes in volume between reporting periods that reflect changes in service delivery from those that are an artefact of changes in data quality.

The analysis in this report is based on the latest data submitted by providers during the two month window to provide data. Any data which are re-submitted by a provider during the submission window will be used in place of an earlier submission within the window. We invite and welcome feedback from users on our constructions.

Reliability

Coverage – are all eligible providers submitting data?

All providers of NHS-funded maternity services should submit MSDS data. However, at present not all providers are making submissions. All trusts made a submission, one independent provider did not. However some trusts submitted a limited number of tables.

This publication reports on activity that was submitted for the April 2019 reporting period. As a dramatically revised national level dataset there are a number of issues in terms of non-response from providers which in turn has an impact on the geographical coverage expected of the dataset hence caution should be taken when interpreting the data at levels higher than individual provider level. Because of this no figures derived from the MSDS data have been presented as England total figures, rather they are presented in terms of all providers who submitted data to the MSDS for the reporting period in question. While the number of submitting providers is all we expect, the number of providers submitting valid data for each data table and data item varies widely. Totals therefore continue to be presented as 'all submitters' values, and users of the data should consider the coverage for the relevant analysis when interpreting the data.

Coverage – is data for all eligible people included in the submission?

Local knowledge may be required to assess the completeness of a submission, based on information about local caseload. This publication provides detailed information about activity and providers and commissioners are encouraged to review this to ensure that submissions accurately reflect the local situation. Providers should also use all the aggregate record counts produced at the point of submission as part of the Data Summary Reports to check coverage in key areas (e.g. number of booking appointments.)

Where an organisation is submitting delivery data to MSDS, the number of births reported in a month can be compared to the 2017-18 monthly average number of births in HES to review the levels of activity reported in the MSDS for each organisation (noting that MSDS is not limited to births in hospital). Where an organisation is not yet submitting delivery data to MSDS, a similar comparison can be made using the number of booking appointments in the month, bearing in mind that not all booking appointments will lead to a birth for that pregnancy within a reporting organisation. 

Duplication

Validations are in place to ensure that there is only one booking appointment for each woman submitted to NHS Digital per organisation for the particular reporting period. It is possible that one woman may have multiple booking appointments recorded for the same pregnancy for different providers. For any calculated total value presented in this report a woman is only counted once in relation to the activity related to the booking appointment. For example, if a woman is reported for a booking appointment by two separate organisations within the same NHS England Region then they would be counted in any total for each of these providers presented at the provider level but would only be counted once for the overall NHS England Region. However, if the woman was reported by two separate providers within two separate NHS England Regions then they would be counted in the overall total calculated for each NHS England Region.

If a woman is reported for a booking appointment by two separate organisations within the same NHS England Region and different data are submitted by each provider then this woman may be counted twice for the overall NHS England Region total. For example, where a woman has an age recorded as 39 by one provider and 40 by another provider this woman would be counted in the NHS England Region total twice, once in the '35 to 39' age group and once in the '40 to 44' age group.

Similarly, a baby reported by two separate organisations within the same NHS England Region with an Apgar score at 5 minutes of 6 recorded by one provider and 8 by another provider would be counted in the NHS England Region total twice, once in the '0 to 6' group and once in the '7 to 10' group.

Timeliness of recording events on local systems

Whilst local systems may be continuously updated, the MSDS submission process provides a time-limited opportunity for data relevant to each month to be submitted. The submission window opens the day after the reporting month and remains open for two months. This means that the timeliness of recording all relevant activity on local systems has an impact on the completeness of the MSDS submission. For example, a booking appointment made in May 2019, but not entered onto the local system until August 2019, will not be included in the May 2019 submission (deadline end of July 2019). Providers should use the data summary reports produced at the point of submission to ensure that all relevant data has been included.

The submission requirements for MSDS are that all appropriate activity (e.g. booking appointment, dating scans, etc.) be included in the submission for each month in which they start, continue or end. It is important that data providers ensure that NHS numbers are submitted consistently because this is a key piece of information for creating the person identifiers in our records.

Quality of Experimental Analysis

It should be noted that these statistics are presently experimental in nature and are likely to be subject to further refinement; reference should be made to all accompanying footnotes and commentary when using these statistics.

Timeliness refers to the time gap between publication and the reference period. Punctuality refers to the gap between planned and actual publication dates.

The monthly publication is based on the most recent available final data. For this publication, the Maternity Services Monthly Statistics report is published within four weeks of the submission window closing.

The Maternity Services Monthly Statistics report for May data was released on the pre-announced publication date and is therefore deemed to be punctual.

Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability is the degree to which data can be compared over time and domain.

Coherence

There is no other monthly publication that includes the same measures as are included in this publication. 

As well as this monthly report on maternity services, NHS Digital has also published the NHS Maternity Statistics 2017-18 report. This annual publication uses Hospital Episode Statistics (HES) data submitted to NHS Digital and has been published annually since 2001-02. The latest report also includes 2017-18 annual data from the MSDS.

Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.

Accessibility

Alongside this background quality document, an executive summary is made freely accessible via the NHS Digital website as an HTML page together with a supporting monthly data file in machine-readable format (with an accompanying metadata document).

Re-use of our data is subject to conditions outlined here: https://digital.nhs.uk/about-nhs-digital/terms-and-conditions

Data Services for Commissioners Regional Offices (DSCROs) can obtain a record level extract of data relevant to the Clinical Commissioning Groups (CCGs) that they support and can share data with these CCGs subject to the relevant data sharing agreements being in place. Information about DSCROs is available from https://digital.nhs.uk/services/data-services-for-commissioners-dsfc

Clarity

The monthly data file is presented as a CSV file, with an accompanying metadata file in MS Excel format. A broad definition of each measure, including the data items used in the analysis and constructions and current or intended uses are provided. Terminology is defined where appropriate.

Full details of the way that MSDS returns are processed, which will be of use to analysts and other users of these data, are provided in the MSDS User Guidance, available on the NHS Digital website: https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets/maternity-services-data-set

In order to prevent disclosure of identities or information about service users, all figures for all organisations which submitted, are rounded to the nearest 5. All figures between 1 and 7 are rounded to 5. 

This refers to the extent to which different aspects of quality are balanced against each other

The format of this publication meets user needs for a greater wealth of information on maternity services in England. Benefits to users include the publication of detailed data on a monthly basis together with associated data quality measures, as well as a visual representation of the national picture on a monthly basis.

The aggregate underlying data provides a much greater scope of analysis and will support a variety of local uses as well as meeting our obligations under the Code of Practice for Statistics and the Transparency Agenda.

The format of this publication balances the need for increased frequency of reporting and scope of analysis with NHS Digital resources and production time. NHS Digital is publishing the data in a new HTML format, whilst supporting the Open Data initiative by also publishing data in a machine-readable format. By publishing a range of clearly defined measures in a timely fashion we hope to support discussions between providers and commissioners about caseload and activity and promote a virtuous cycle of improving data quality, through use.

For MSDS version 2.0 analysis, we have stopped producing the basic suite of data tables as part of this publication. However, the machine-readable data file is very detailed and allows data users to easily produce custom tabulations as required for their own analytical purposes.

This refers to the processes for finding out about users and uses, and their views on the statistical products.

The purpose of the MSDS monthly reports is to provide maternity service providers, commissioners and other stakeholders with timely information about activity. This is intended to support changes in commissioning arrangements as services move from block commissioning to commissioning based on activity and outcomes for mothers and babies.

For members of the public, researchers and other stakeholders, the release provides up to date information about the people in contact with services.

NHS Digital held a number of workshops for maternity service providers and system suppliers, providing updates on the development of the data set and allowing clinicians, system administrators and informatics staff to provide feedback during the development stage. NHS Digital continues to hold regular workshops with service providers and system suppliers, providing updates on data quality and future development of the dataset, and obtaining feedback on use of the data.

NHS Digital has also held workshops with analysts who use MSDS data and other maternity data to seek feedback to inform further development of the monthly reports and other outputs and encourage wider use of MSDS data.

This refers to the effectiveness, efficiency and economy of the statistical output.

As a 'secondary uses' data set, the MSDS does not require the collection of new data items by maternity providers. It re-uses existing clinical and operational data for purposes other than direct patient care.

Providers are not required to submit data held only on paper records as no provision has been made in the MSDS for the cost of transcribing these records to an electronic format.

Only three of the data tables are required to flow in every MSDS submission (MSD001, MSD002 and MSD101); completion of the remaining tables is only necessary when activity has occurred that is captured within these tables.

This refers to the procedures and policy used to ensure sound confidentiality, security and transparent practices.

All publications are subject to a standard NHS Digital risk assessment prior to issue. Disclosure control is implemented where deemed necessary.

Please see links below to relevant NHS Digital policies: