The proximity between an estimate and the unknown true value
The accuracy of the SALT and ASC-FR data is the responsibility of the local authorities who submit the data to NHS Digital. Both returns are aggregate collections taken from administrative systems. As NHS Digital does not have access to the individual records behind the aggregate counts, we are reliant on local authorities to assess their own data quality.
In many instances, assessing reliability depends on local knowledge, as each local authority determines the approach taken in their area. What may be an anomaly in one area could be considered standard practice elsewhere. However, a range of activities are undertaken (outlined in more detail below) to check and improve data quality. Discussions with local authorities have provided useful anecdotal information about distributions and trends.
The submission and validation process for each collection is carried out as follows:
- The local authority collates the data for submission in the relevant collection form. This form includes inbuilt validations to allow local authorities to check their data for common issues prior to submission.
- Local authority submits data by the mandated deadline.
- All local authorities who met this deadline receive a data quality report covering critical validations and providing some derived totals allowing local authorities to confirm their data is correct.
- NHS Digital reviews the quality of all files submitted and may provide additional support to local authorities with significant data quality issues.
- Local authorities can resubmit data to amend any identified quality issues.
- Final deadline for submission.
- Following the final deadline NHS Digital will carry out analysis of the quality of final data. Although local authorities cannot resubmit data after this point, they may be contacted for additional clarification or context.
The validation checks carried out throughout this process included:
- Validations across worksheets to check figures across different tables that store the same data.
- Checks for missing data items.
- Checks for instances of recorded activity without associated expenditure and vice versa.
- Comparing the unit costs with those from other local authorities and flagging outliers.
- In ASC-FR, planned Better Care Fund expenditure was provided to local authorities to compare their Income from Better Care Fund value against.
The final validation stage consisted of looking at the responses to the validation checks mentioned above to see if there are clear instances where data submitted is implausible or local authorities have submitted data not in line with the guidance for the data collection.
We approached local authorities for additional context where outliers or inconsistencies were identified. Not everyone was able to respond and so the data quality summary reflects all explanations received from local authorities, either at the point of submission or in response to our follow-up queries.
Common issues local authorities advised us of include:
In SALT
- Limitations with case management systems for reporting of some data items, particularly around full cost clients.
- Several local authorities have mentioned difficulties obtaining and/or quality assuring data from third party organisations such as mental health or carer data.
In ASC-FR
- Allocating expenditure by Primary Support Reason (PSR) in the ASC-FR can be challenging as many finance systems are not configured to collect this data.
- The allocation of Better Care Fund (BCF) and improved Better Care Fund (iBCF) in the ASC-FR has improved in 2018-19 compared to previous years, so care should be taken when comparing to historic data.
Full details of issues that NHS Digital are aware of are included in the accompanying Data Quality Summary.
Information on the number of local authorities returning data at the mandated and final deadlines across both collections is available on the completeness tab of the accompanying Data Quality Summary.
We have now been advised that due to technical issues experienced, the data for East Riding does not represent their finalised position on permanent admissions.
