The proximity between an estimate and the unknown true value
The accuracy of the DPA data is the responsibility of the local authorities who submit the data to NHS Digital. The return is an aggregate collection taken from administrative systems. As NHS Digital does not have access to the individual records behind the aggregate counts, we are reliant on local authorities to assess their own data quality.
In many instances, assessing reliability depends on local knowledge, as each local authority determines the approach taken in their area; what may be an anomaly in one area could be considered standard practice elsewhere. However, a range of activities are undertaken (outlined in more detail below) to check and improve quality.
The submission and validation process for each collection is carried out as follows:
- The local authority collates the data for submission in the relevant collection form. This form includes inbuilt validations to allow councils to check their data for common issues prior to submission.
- The local authority submits data by the mandated deadline.
- All local authorities who met the deadline receive a data quality report covering critical validations and also providing some derived totals allowing councils to confirm their data is correct.
- NHS Digital reviews the quality of all files submitted and may provide additional support to local authorities with significant data quality issues.
- Local authorities are able to resubmit data to amend any identified quality issues.
- Final deadline for submission.
- Following the final deadline NHS Digital will carry out analysis of the quality of final deadline. Although local authorities cannot resubmit data after this point, they may be contacted for additional clarification or context.
The validation checks carried out throughout this process include:
- Checks for missing data items
- Checks for instances of recorded activity without associated expenditure and vice versa
- Validations across worksheets to check figures across different tables store the same figures i.e. For DPA003 table 3a the number of new requests where a DPA was approved should match the number of new DPAs recorded in DPA001.
The final validations consisted of looking at the responses to the validation checks mentioned above to see if there are clear instances where data are implausible or local authorities have submitted data not in line with the guidance for the data collection. Local authorities are then contacted where necessary.
Common issues local authorities advised us of include:
- Reconciling the total number of new DPAs against those captured in a more granular breakdown.
- 2017/18 submissions no longer accurately reflecting the previous year’s position, meaning year on year comparisons are not recommended.
Information on the number of Local Authorities returning data by the first cut (by the mandated deadline) and by the final cut (by the final deadline) is available on the completeness tab of the Data Quality summary.