Skip to main content

Publication, Part of

Community Services Statistics, September 2021

Experimental statistics

Data quality statement

Introduction

Community Services that are publicly funded are required to submit data relating to their clinical activity in the CSDS. The CSDS does not cover non-publicly funded community services provided by independent organisations (e.g. private clinics).

Providers receive immediate feedback on the quality of their submission and a variety of data quality checks are run as part of the validation and load process for the monthly data, prior to production of this monthly release.

However, as coverage and accuracy improve, these monthly reports will also evolve to report on different measures.

These statistics are classified as experimental and should be used with caution. Experimental statistics are new official statistics undergoing evaluation. They are published in order to involve users and stakeholders in their development and as a means to build in quality at an early stage. More information about experimental statistics can be found on the UK Statistics Authority website.

 

Important information for July 2020 data onwards

The Community Services Monthly Statistics reports for July 2020 onwards are the first to come from the new version of the Community Services Data Set (CSDS v1.5).

This version of the data set has had some changes and additions, to field names, table names and the clinical coding we receive. Our collection portal has also been redeveloped, to enable consistency in submissions and efficiency in analysis and data linkage in the fullness of time. System suppliers are at different stages on developing their new solution and delivering that to providers. In some cases, this has limited the aspects of data that could be submitted to NHS Digital.

These changes, in conjunction with the COVID-19 pandemic, have meant that we have not received as much data from every provider as we did in early 2020 under the previous version of the data set. Some tables are mandatory for upload. Whilst these tables are mandatory, not all are required for an upload to be successful. We expect their completeness to improve over time, as they did with the previous version of the CSDS and we are looking at ways of supporting the improvements.

149 providers submitted data for September 2021.


Purpose of document

This page aims to provide users with an evidence-based assessment of the quality of the statistical output of the Community Services Statistics publication by reporting against those of the nine European Statistical System (ESS) quality dimensions and principles appropriate to this output.

 

In doing so, this meets NHS Digital’s obligation to comply with the UK Statistics Authority (UKSA) code of practice for statistics and the following principle in particular:

 

  • “Producers of statistics and data should explain clearly how they assure themselves that statistics and data are accurate, reliable, coherent and timely.”

For each dimension, this page describes how this applies to the publication and references any measures in the accompanying monthly data quality measures report that are relevant for assessing the quality of the output.


Assessment of user needs and perceptions

This dimension covers the processes for finding out about users and uses and their views on the statistical products. 

The CSDS has been developed as a key driver to achieving better outcomes of community care. These monthly reports provide community service providers, commissioners and other stakeholders with timely information about activity. This is intended to support changes in commissioning arrangements as services move from block commissioning to commissioning based on activity and outcomes.

Monthly reports from the CSDS include:

  • An HTML executive summary
  • Two CSV files containing the aggregate underlying data
  • A CSV data quality file showing national and provider-level data quality measures
  • A detailed metadata file describing all of the measures in the analysis. This includes, for each measure, how it has been constructed from providers' submissions and how and where it is used.

NHS Digital has held and continues to hold workshops for community service providers and system suppliers, providing updates on the development of the reports and allowing clinicians, system administrators and informatics staff to provide feedback on reporting during development and beyond.

Reporting is currently shaped by the coverage and accuracy of the data, however as data quality improves these reports will change to present other information. Feedback is very welcome via our [email protected]  address (please quote ‘CSDS Monthly Statistics’ in the subject line).


Relevance - Content and Coverage

This dimension covers the degree to which the statistical product meets user needs in both content and coverage.

This publication comprises a set of reports which have been produced from publicly funded community service providers’ monthly CSDS submissions.

The CSDS has been developed to help achieve better community services outcomes. It provides data that will be used to commission services in a way that improves health, reduces inequalities and supports service improvement and clinical quality. This is done by capturing detailed information about the individual (gender, ethnicity, age deprivation etc.) along with associated risk factors, such as whether they are on a child protection plan and other safeguarding indicators and reporting against this their referral details to publicly funded community services along with any information around care contacts and outcomes that have been achieved as a result of this activity. Further detail on the information captured within the dataset is available from https://www.digital.nhs.uk/Community-Services-Data-Set.

Organisations that provide publicly-funded community services are required to report to the CSDS data set. However, different models of local health service provision across England means that the organisations delivering such activity varies between local areas. For example, some regions have a dedicated Community Healthcare trust for all ages of patients ,whilst others may separate community healthcare for adults and children between separate organisations and for other organisations responsibilities for such activities may fall within a local acute trust. Such models of provision can be fluid and therefore, it is difficult to determine an exact number of organisations that should be submitting data to the CSDS.

Therefore, figures at levels other than individual provider level may not fully represent the true figure at that level. As a result of this, no figures derived from the CSDS data have been presented as England total figures, rather they are presented in terms of all providers who submitted data to the CSDS for the reporting period in question.

There are currently 36 tables in the CSDS data set that each contain information relating to a specific event or type of information that may occur during community care. However, only 6 of these tables are currently reported on. This will change to include more data tables as data quality and coverage improves and hence increase the relevance of this report to reflect wider user needs.

Local knowledge may be required to assess the completeness of a submission, based on information about local caseload. This publication provides detailed information about activity, which providers and commissioners are encouraged to review to ensure that submissions accurately reflect the local situation. Providers should also use all the aggregate record counts produced at the point of submission as part of the Submission Summary Reports to check coverage in key areas (e.g. Number of referrals).


Accuracy and reliability

This dimension covers, with respect to the statistics, their proximity between an estimate and the unknown true value.

Accuracy is the proximity between an estimate and the unknown true value.

The CSDS is a rich, person level data set that records packages of care received by individuals in contact with NHS-funded community services. NHS Digital provides a number of different reports at different stages in the data flow to ensure that the submitted data reflects the services that have been provided:

For data suppliers only:

At the point of submission:

  • Providers receive immediate feedback on the quality of their submission through a validation file. This file includes record-level reports of any submission errors, giving the data providers detailed information about which records produced which errors.

On receipt of processed data by NHS Digital:

  • A variety of data quality checks are run as part of the validation and load process for monthly data, prior to production of this monthly release. Where there are concerns about data quality, we contact providers directly so that any issues with local data extraction processes can be addressed for a future submission. 

For all users:

As part of this publication, we publish a CSV file of data quality of submissions from community service providers.

Users of the data must make their own assessment of the quality of the data for a particular purpose, drawing on these resources.

 

Reliability is the closeness of early estimates to subsequent estimated values.

Reliability is the closeness of early estimates to subsequent estimated values.

Whilst local systems may be continuously updated, the CSDS submission process provides two opportunities for data relevant to each month to be submitted. The primary submission window opens roughly one month following the end of the reporting month and the refresh submission window opens roughly two months following the end of the reporting month. This means that the timeliness of recording all relevant activity on local systems has an impact on the completeness of the CSDS submission. For example, a referral made in September 2020, but not entered onto the local system until the beginning of December 2020, will not be included in the final September 2020 submission. Providers should use the validation reports produced at the point of submission to ensure that all relevant data has been included. Statistics published in this report are calculated using data taken after the refresh submission window has closed with no early estimated values published.


Quality of experimental analysis

It should be noted that these statistics are presently experimental in nature and are likely to be subject to further refinement; reference should be made to all accompanying footnotes and commentary when using these statistics..


Timeliness and punctuality

Timeliness refers to the time gap between publication and the reference period. Punctuality refers to the gap between planned and actual publication dates.

 

Timeliness of recording events on local systems

Whilst local systems may be continuously updated, the CSDS submission process provides two opportunities for data relevant to each month to be submitted. The submission window opens the day after the reporting month and remains open for two months. This means that the timeliness of recording all relevant activity on local systems has an impact on the completeness of the CSDS submission. For example, a referral made in May 2020, but not entered onto the local system until August 2020, will not be included in the May 2020 submission (deadline end of July 2020). Providers should use the data summary reports produced at the point of submission to ensure that all relevant data has been included.

The submission requirements for CSDS are that all appropriate activity (e.g. referrals, care contacts, care activities etc.) be included in the submission for each month in which they start, continue or end. It is important that data providers ensure that NHS numbers are submitted consistently because this is a key piece of information for creating the person identifiers in our records.

The analysis in this report is based on the latest data submitted by providers during the two months-long windows to provide data. Any data which are re-submitted by a provider during the submission window will be used in place of an earlier submission within the window. We invite and welcome feedback from users on our constructions.

For this publication, the Community Services Monthly Statistics report is published within 6 weeks of the submission window closing.

The Community Services Monthly Statistics report for September 2021 data was released on the pre-announced publication date and is therefore deemed to be punctual.


Coherence and comparability

Coherence is the degree to which data which have been derived from different sources or methods but refer to the same topic are similar. Comparability is the degree to which data can be compared over time and domain. 

There is no other monthly publication that includes the same measures as are included in this publication. 

However, for different activities that will be reported within the CSDS, there are a number of existing national data collections and reports that currently publish data depending upon their own specific topical area these are listed below.

Vaccination and Immunisation Data

Information on childhood immunisation coverage at ages one, two and five are collected through the UK Cover of Vaccination Evaluated Rapidly (COVER) collection by Public Health England. These aggregated data are collected from Child Health Information Systems (CHISs), which are computerised systems storing clinical records supporting health promotion and prevention activities for children, including immunisation.

Data from the COVER is published by NHS Digital. The latest statistics are available via the following link https://digital.nhs.uk/data-and-information/publications/statistical/nhs-immunisation-statistics

In October 2017, an annual report was released comparing the number of childhood vaccinations recorded in the CYPHS to the number recorded by the COVER programme for 2016-17. This comparison was intended to highlight areas for improvement in data coverage and completeness for the CYPHS data set and is available via the following link: http://digital.nhs.uk/catalogue/PUB30103

Further information on vaccination and immunisation is captured by NHS Digital as part of the General Practice Extraction Service (GPES) via which GP Contract Services data are collected. This represents recorded activity undertaken by a practice. It is important to note that not all practices are required to flow vaccination and immunisation data for this. Currently information on vaccination and immunisations from GPES is published as Experimental Statistics and are available via the following link:

https://digital.nhs.uk/data-and-information/publications/statistical/gp-contract-services

 

National Child Measurement Programme

The National Child Measurement Programme (NCMP) measures the height and weight of children in reception class (aged 4 to 5 years) and year 6 (aged 10 to 11 years) to assess overweight and obesity levels in children within primary schools. This data can be used at a national level to support local public health initiatives and inform the local planning and delivery of services for children.

Local Authorities are asked to collect data on children's height and weight from all state maintained schools within their area. Participation in the programme is not compulsory, but non-participation is on an opt-out basis only. The data are submitted to NHS Digital and all of the returns are collated and validated centrally.

The latest published data on the NCMP is available via the following link: https://digital.nhs.uk/areas-of-interest/public-health/national-child-measurement-programme

 

Children's public health 0-5 years - national reporting

Public Health England (PHE) implemented an interim approach to national reporting on children's public health for those aged 0-5 years in 2015/16. This was until such time as the Maternity and Children’s Datasets of which the CSDS is one have reached full maturity in terms of data quality and coverage.

Data is submitted to PHE by local authorities on a voluntary basis and is published quarterly. The indicators include breastfeeding prevalence at 6-8 weeks and Health Visitor Service Delivery Metrics described in legislation for universal health visitor reviews and key outcomes for children resident within a local authority (based on the child's postcode).

Data on both Breastfeeding at 6 to 8 weeks after birth and Health Visitor Service Delivery Metrics are available from https://www.gov.uk/government/collections/child-and-maternal-health-statistics

An analysis of CSDS data focusing on 6-8 week breastfeeding status and 24, 27 and 30 month Ages and Stages (ASQ-3) scoring is now included as part of the CSDS publication releases on a monthly basis. The previously published quarterly reports are available via the following link:

https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets/community-services-data-set/community-services-data-set-supplementary-reports

Community Information Data Set (CIDS)


The CIDS is  a patient level, output based, secondary uses data set which will deliver robust, comprehensive, nationally consistent and comparable person-based information on patients who are in contact with Community Services. Currently CIDS has only been mandated for local data collection and data extraction only. CIDS is aligned with the CSDS data set, the two share the same structure and data items.


As such there may be local reporting of data using CIDS which would report on similar data to that reported in the CSDS. However, the nature and extent of such local reporting remains unknown at the national level. 

 


Accessibility and clarity

Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.

The format of this publication meets user needs for a greater wealth of information on community services in England. Benefits to users include the publication of detailed data on a monthly basis, together with associated data quality measures, as well as a visual representation of the national picture on a monthly basis.

We produce a machine-readable data file which allows data users to easily produce custom tabulations as required for their own analytical purposes.

This report is made freely accessible via the NHS Digital website as an HTML report together with a supporting monthly data file in machine-readable format (with an accompanying metadata document).

The monthly data files are also available on the data.gov website, here: https://data.gov.uk/data/search

Re-use of our data is subject to conditions outlined here: https://digital.nhs.uk/about-nhs-digital/terms-and-conditions

Providers are able to obtain a record level data extract for their care activity from the Strategic Data Collection Service (SDCS) Cloud. 

The monthly data file is presented as a .csv file, with an accompanying metadata file in MS Excel format. A broad definition of each indicator, including the data items used in the analysis and constructions and current or intended uses are provided. Terminology is defined where appropriate. Full details of the way that CSDS returns are processed, which will be of use to analysts and other users of these data, are provided in the CSDS User Guidance, available on the NHS Digital website: https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets/community-services-data-set


Trade-offs between output quality components

This dimension describes the extent to which different aspects of quality are balanced against each other. 

The format of this publication meets user needs for a greater wealth of information on publicly-funded community services in England. Benefits to users include the publication of detailed data on a monthly basis together with associated data quality measures, as well as a visual representation of the national picture on a monthly basis.

The aggregate underlying data provides a much greater scope of analysis and will support a variety of local uses as well as meeting our obligations under the Code of Practice for National Statistics and the Transparency Agenda.

The format of this publication balances the need for increased frequency of reporting and scope of analysis with NHS Digital resources and production time. NHS Digital is supporting the Open Data initiative by also publishing data in a machine-readable format. By publishing a range of clearly defined dimensions and measures in a timely fashion, we hope to support discussions between providers and commissioners about caseload and activity and promote a virtuous cycle of improving data quality through use.


Balance between performance, cost and respondent burden

This dimension describes the effectiveness, efficiency and economy of the statistical output.

As a 'secondary uses' data set, the CSDS does not require the collection of new data items by community providers. It re-uses existing clinical and operational data for purposes other than direct patient care.

Providers are not required to submit data held only on paper records as no provision has been made in the CSDS for the cost of transcribing these records to an electronic format.

Only two of the data tables are mandated to flow each time any activity is reported within the CSDS (CYP001; patient demographics and CYP002; GP registration). Completion of the remaining tables is only required when activity has occurred that is captured within these tables.


Confidentiality, Transparency and Security

The procedures and policy used to ensure sound confidentiality, security and transparent practices. 

For national level data, no disclosure control is required for small numbers.

For counts below national level:

1) If the national total is between 1 and 7 (inclusive), no sub-national breakdown will be displayed.

2) If the national total is greater than or equal to 8 a. Sub-national counts between 1 and 7 (inclusive) will be displayed as ’*’. b. Zeroes will be unchanged. c. All other counts will be rounded to the nearest 5.

For percentages below national level:

1) Where the numerator or denominator is between 1 and 7 (inclusive), no percentage or rate is calculated, and a ’*’ will be displayed.

2) Where the numerator is zero, the percentage will be 0%.

3) Where the unrounded numerator and denominator are greater than or equal to 8, a percentage or rate is calculated using the rounded numerator or denominator.

 

All publications are subject to a standard NHS Digital risk assessment prior to issue. Disclosure control is implemented where deemed necessary.

Please see links below to relevant NHS Digital policies:


Last edited: 11 January 2022 8:55 am