This dimension covers, with respect to the statistics, their proximity between an estimate and the unknown true value.
The accuracy and reliability of the dataset underpinning the analyses in the report is ensured by a rigorous validation procedure. Further details are provided in Validation of National Child Measurement Programme data.
As records are submitted, the NCMP system checks that all mandatory data items have been provided and data validation rules have been met.
- Records with missing data items are rejected.
- Invalid data items (e.g. incorrect ethnicity codes) are rejected.
- Unexpected data items (e.g. “extreme” heights) generate warning flags that require LA confirmation.
The NCMP system provides LAs with real-time data quality indicators throughout the collection period. Indicators breaching the data quality thresholds are highlighted. This enables LAs to monitor the quality of their data during the collection period and take action if necessary. The data quality thresholds are provided in the validation document mentioned above. The LA’s NCMP Lead is required to sign off these indicators as part of finalising their data at the end of the collection. Indicators breaching the thresholds require the NCMP Lead to provide a breach reason before it is possible to finalise the submission.
The performance of LAs against these data quality measures is published in Data Quality Table A alongside the national report.
In recognition of the effect of natural year to year variation, confidence intervals are included around the prevalence estimates in the tables and these should be considered when interpreting results. A confidence interval gives an indication of the sampling error around the estimate calculated and takes into consideration the sample sizes and the degree of variation in the data. They are used to determine whether any differences in prevalence figures are likely to be real or due to natural variation.
The sample sizes and participation rates for NCMP are usually large (over one million children measured and a participation rate exceeding 90 per cent for the 2008/09 to 2018/19 collections), so the 95 per cent confidence intervals for prevalence estimates at national level are very narrow. The 2019/20 and 2020/21 collections yielded fewer measurements so have wider, albeit still narrow, confidence intervals (see Table 2 of the report tables). This indicates a small margin of potential error. The comparisons that feature in this report have all been tested at a 95 per cent significance level. Where two figures are described as being different (e.g. higher/lower or increase/decrease etc.) the result of the test has determined a statistically significant difference. Further details are provided in appendix E of the publication.
As the data are based on a sample (rather than a census) of pupils, the estimates are subject to sampling error. Appendix D details how to calculate sampling errors for this survey, and the excel tables include true standard errors and design effects calculated for key survey estimates.
In general, attention is drawn to differences between estimates only when they are significant at the 95% confidence level, thus indicating that there is less than 5% probability that the observed difference could be due to random sampling variation when no difference occurred in the population from which the sample is drawn.
The limitations of the survey estimates are discussed in appendix C.