Skip to main content

Publication, Part of

Autism Waiting Time Statistics - Quarter 1 (April to June) 2018-19 to Quarter 4 (January to March) 2019-20

Experimental statistics,

Current Chapter

Limitations of the statistics and interpretation issues


Limitations of the statistics and interpretation issues

What the data can tell you

These measures contribute to a picture of overall waiting times for autism spectrum disorder (ASD) diagnostic pathways within mental health services. These waiting times are for those referred for suspected autism to their first care contact in the forward model and how long those diagnosed with autism waited for a diagnosis in the reverse model.

What the data cannot tell you

There are known data quality limitations with respect to the completeness of care contacts and specific information related to these, such as the specific team that referred the patient or the presence of formal diagnoses. For these reasons, these statistics do not yet represent a complete picture of autism diagnostic pathways.

In the forward model, for any referrals not having a first care contact it is not possible to determine how long such referrals have been waiting. This is because the first contact may have taken place in a yet to be submitted reporting period.

What the data can and cannot tell you is laid out in more detail below.

Autism service providers

Work to determine which provider organisations should be submitting data for people with autism is ongoing and as such a finalised list of in-scope providers is not yet available. In the absence of an in-scope list of autism service providers, we are unable to state how many of the organisations that have submitted data to MHSDS should have submitted autism data.

We are also unable to determine the volume of referrals or diagnoses for autism in child development services. These services are out of scope of the MHSDS and as such their data is unavailable to us. It is important to note that the majority of children assessed for autism in the United Kingdom are seen in these out of scope services. This will mean our figures will underestimate prevalence and the associated impact on health services.

Submitting autism data

The submission of autism data can be complex as the process of diagnosing an individual person involves multiple multi-disciplinary teams and can take time to agree a diagnosis of autism. As a result of this, the patient journey can be difficult to accurately represent in the Mental Health Services Data Set.

Further complexity can arise when some patients have more than one health condition and can be assessed for different conditions under the same referral. This is then also difficult to record and measure over time and may result in data submission issues.

Autism waiting time information relies on submitters providing referral, care contact and diagnosis information in a timely manner; ideally this would be by the time the submission window closes for each reporting period. If a provider submits any activity data (referral or diagnosis information) related to the reporting period outside of the submission window, then these are accepted into the dataset if the referral is still open but may not feature in official statistics reports, as official statistics are taken at specific intervals. Care contact information cannot be updated retrospectively for any open referrals due to the limitations of the dataset version and the requirements of submitters to send this information promptly. As a result of this, there are records with a missing care contact date and so waiting times between referral and first care contact cannot be derived for some patients within the forward model.

A missing care contact date can mean one of three things:

1. A care contact/first appointment occurred but was not submitted before the submission window closed
2. The person is in fact still waiting for a first appointment
3. The referral was discharged without a first appointment having taken place

It is not possible to discern the underlying reason from the dataset currently

Impact of introducing 'suspected autism' as a primary reason for referral

Until 1 April 2018, there was no way to select ‘suspected autism’ (referral code 25) as a primary reason for referral. Instead it was combined within a broader category with various other health conditions under ‘neurodevelopmental conditions’ (referral code 24).

Even from 1 April 2018, some submitters may still be submitting autism referrals as referral code 24 if they are not aware of the new referral reason explicitly for ‘suspected autism’.

As we are unable to identify these as suspected autism records, we are unable to use these records for waiting times between referral and first care contact within the forward model.

However in the reverse model we are looking at all autism diagnosis irrespective of the reason for referral and a breakdown by reason for referral is provided in the statistics.


Forward model specific interpretation issues

As time has elapsed since the initial publication of the forward model, the number of complete journeys from referral to first care contact has increased. This gives a more complete picture of the waiting times between referral and care contact. However, there is still a reducing number of complete journeys as you move towards the most recent data submission periods. There is also a gap identified in this model in that it will not pick up those referrals coded other than for suspected autism, irrespective of whether those referrals resulted in an autism diagnosis.

The main limitation of the forward model is that there is a reliance on future data to identify care contacts attached to referrals. This is less of an issue in older records. For example, in quarter one of the 2018-19 financial year, a larger proportion of the referrals recorded have then gone on to have a care contact recorded. For more recent referrals, for example those recorded in quarter three of the 2019-20 financial year, a smaller proportion of the referrals recorded have then gone on to have a care contact recorded.

At the point at which the analysis was undertaken, the latest available full quarter of data is January – March 2020 (quarter four 2019-20) and a large proportion of referrals received in this quarter have not had enough time elapsed to assess whether they meet the 13 week waiting time target to first care contact. As a result of this, only quarter one to quarter three of 2019-20 are published.

The above chart shows the percentage of referrals that have a care contact recorded across the reporting periods.

National total figures include referrals for people whose demographics are not known, as such the sum of the individual categories may be less than the total.


Reverse model specific interpretation issues

While the reverse model is more robust in that it has no reliance on future data; instead taking existing diagnosis records and looking back to when the person was first referred, there are some existing interpretation issues.

We expect the diagnosis date recorded for a referral to be on or after the initial referral request received date. However, there are a few records submitted with a diagnosis date before referral date and we have excluded such records from our analysis.

There are a few providers submitting autism diagnosis data for the same referral with different diagnosis date over different reporting periods. In our analysis we are checking for each referral with autism diagnosis in a reporting period that there are no earlier diagnoses recorded in any of the previous reporting months and including only those which have an autism diagnosis recorded for the first time in the reporting period.

National total figures include diagnosis records for people whose demographics are not known, as such the sum of the individual categories may be less than the total.


Review

The statistics presented in this publication are currently designated as Experimental Statistics and as such remain under constant review, although major changes to methodologies are made between reporting years in order to preserve time-series. 

The Mental Health Analysis team welcomes any comments or feedback on the publication so please send any such communications to [email protected]  with ‘Autism Waiting Time Statistics’ in the subject field.


Last edited: 6 July 2021 9:33 am