Skip to main content

Publication, Part of

Autism Waiting Time Statistics - Quarter 2 2019-20 to Quarter 1 2020-21 and Quarter 2 (July to September) 2020-21

Experimental statistics,

Current Chapter

Limitations of the statistics and interpretation issues


Methodological change in calculation of waiting time rates

The following change affects only the rate of completion of an initial care contact within a specified time for individuals with a referral record for suspected autism in MHSDS.

In August and September's releases, the waiting time measures ASD03, ASD04, ASD05, ASD06, ASD07 and ASD08 were all calculated from the denominator ASD02 (number of referrals for suspected autism in the reporting period that have a first care contact recorded). This has now changed and the above listed measures are now calculated from the denominator ASD01 (number of referrals for suspected autism in the reporting period).

10 December 2020 09:30 AM

Limitations of the statistics and interpretation issues

What the data can tell you

These measures contribute to a picture of overall waiting times for autism spectrum disorder (ASD) diagnostic pathways within mental health services. These waiting times are for those referred for suspected autism to their first care contact in the forward model and how long those diagnosed with autism waited for a diagnosis in the reverse model.

What the data cannot tell you

There are known data quality limitations with respect to the completeness of care contacts and specific information related to these, such as the specific team that referred the patient or the presence of formal diagnoses. For these reasons, these statistics do not yet represent a complete picture of autism diagnostic pathways.

In the forward model, for any referrals not having a first care contact it is not possible to determine how long such referrals have been waiting. This is because the first contact may have taken place in a yet to be submitted reporting period.

What the data can and cannot tell you is laid out in more detail below.

Autism service providers

Work to determine which provider organisations should be submitting data for people with autism is ongoing and as such a finalised list of in-scope providers is not yet available. In the absence of an in-scope list of autism service providers, we are unable to state how many of the organisations that have submitted data to MHSDS should have submitted autism data.

We are also unable to determine the volume of referrals or diagnoses for autism in child development services. These services are out of scope of the MHSDS and as such their data is unavailable to us. It is important to note that the majority of children assessed for autism in the United Kingdom are seen in these out of scope services. This will mean our figures will underestimate prevalence and the associated impact on health services.

Submitting autism data

The submission of autism data can be complex as the process of diagnosing an individual person involves multiple multi-disciplinary teams and can take time to agree a diagnosis of autism. As a result of this, the patient journey can be difficult to accurately represent in the Mental Health Services Data Set.

Further complexity can arise when some patients have more than one health condition and can be assessed for different conditions under the same referral. This is then also difficult to record and measure over time and may result in data submission issues.

Autism waiting time information relies on submitters providing referral, care contact and diagnosis information in a timely manner; ideally this would be by the time the submission window closes for each reporting period. If a provider submits any activity data (referral or diagnosis information) related to the reporting period outside of the submission window, then these are accepted into the dataset if the referral is still open but may not feature in official statistics reports, as official statistics are taken at specific intervals. Care contact information cannot be updated retrospectively for any open referrals due to the limitations of the dataset version and the requirements of submitters to send this information promptly. As a result of this, there are records with a missing care contact date and so waiting times between referral and first care contact cannot be derived for some patients within the forward model.

A missing care contact date can mean one of three things:

1. A care contact/first appointment occurred but was not submitted before the submission window closed
2. The person is in fact still waiting for a first appointment
3. The referral was discharged without a first appointment having taken place

It is not possible to discern the underlying reason from the dataset currently

Impact of introducing 'suspected autism' as a primary reason for referral

Until 1 April 2018, there was no way to select ‘suspected autism’ (referral code 25) as a primary reason for referral. Instead it was combined within a broader category with various other health conditions under ‘neurodevelopmental conditions’ (referral code 24).

Even from 1 April 2018, some submitters may still be submitting autism referrals as referral code 24 if they are not aware of the new referral reason explicitly for ‘suspected autism’.

As we are unable to identify these as suspected autism records, we are unable to use these records for waiting times between referral and first care contact within the forward model.

However in the reverse model we are looking at all autism diagnosis irrespective of the reason for referral and a breakdown by reason for referral is provided in the statistics.


Forward model specific interpretation issues

As time has elapsed since the initial publication of the forward model, the number of complete journeys from referral to first care contact has increased. This gives a more complete picture of the waiting times between referral and care contact. However, there is still a reducing number of complete journeys as you move towards the most recent data submission periods. There is also a gap identified in this model in that it will not pick up those referrals coded other than for suspected autism, irrespective of whether those referrals resulted in an autism diagnosis.

The main limitation of the forward model is that there is a reliance on future data to identify care contacts attached to referrals. This is less of an issue in older records. For example, in quarter two of the 2019/20 financial year, a larger proportion of the referrals recorded have then gone on to have a care contact recorded. For more recent referrals, for example those recorded in quarter one of the 2020/21 financial year, a smaller proportion of the referrals recorded have then gone on to have a care contact recorded.

At the point at which the analysis was undertaken, the latest available full quarter of data is July – September 2020 (quarter two 2020/21) and a large proportion of referrals received in this quarter have not had enough time elapsed to assess whether they meet the 13 week waiting time targets to first care contact. As a result of this, only quarter two 2019/20 to quarter one 2020/21 are published.

The above chart shows the percentage of referrals that have a care contact recorded across the reporting periods.

National total figures include referrals for people whose demographics are not known, as such the sum of the individual categories may be less than the total.


Reverse model specific interpretation issues

While the reverse model is more robust in that it has no reliance on future data; instead taking existing diagnosis records and looking back to when the person was first referred, there are some existing interpretation issues.

We expect the diagnosis date recorded for a referral to be on or after the initial referral request received date. However, there are a few records submitted with a diagnosis date before referral date and we have excluded such records from our analysis.

There are a few providers submitting autism diagnosis data for the same referral with different diagnosis date over different reporting periods. In our analysis we are checking for each referral with autism diagnosis in a reporting period that there are no earlier diagnoses recorded in any of the previous reporting months and including only those which have an autism diagnosis recorded for the first time in the reporting period.

National total figures include diagnosis records for people whose demographics are not known, as such the sum of the individual categories may be less than the total.


Review

The statistics presented in this publication are currently designated as Experimental Statistics and as such remain under constant review, although major changes to methodologies are made between reporting years in order to preserve time-series. 

The Mental Health Analysis team welcomes any comments or feedback on the publication so please send any such communications to [email protected]  with ‘Autism Waiting Time Statistics’ in the subject field.


Last edited: 6 July 2021 9:35 am