Coherence is the degree to which data which have been derived from different sources or methods but refer to the same topic are similar. Comparability is the degree to which data can be compared over time and domain.
Survey topic content
The first survey in the series, carried out in 1982, measured the prevalence of smoking among pupils and described their smoking behaviour. Trends in smoking were monitored by similar surveys carried out every two years. Questions on alcohol consumption were added to the survey in 1988. The 1998 survey was the first to include questions on the prevalence of drug use.
Break in time series: change to alcohol questions 2016
The question wording which is used both for the “ever drunk alcohol” prevalence indicator and as a filter question for further questions about alcohol use changed from 2016. Previously the question wording was “Have you ever had a proper alcoholic drink – a whole drink, not just a sip? Please don’t count drinks labelled low alcohol?”
However, during cognitive testing of the survey pupils expressed confusion around the terms “proper alcoholic drink” and “low alcohol”. Some pupils reported excluding alcopops and cocktails containing alcohol as they generally tasted of fruit rather than alcohol. As a result, it was decided to remove those terms and therefore the question became “Have you ever had an alcoholic drink - a whole drink, not just a sip?”.
Therefore, whilst this change of wording will deliver a better estimate of the number of children who do drink alcohol it does mean that the results from 2016 onwards are not comparable with previous years. The chapters which include estimates based on the alcohol questions and the tables they are based on have been annotated to mention this. To a lesser extent, this may also affect estimates produced from other alcohol related questions. This is because a slightly wider group of children will now answer these questions, who may have been filtered out of the further alcohol questions based on the previous wording.
Break in time series: mean number of units drunk on each drinking day (part 5)
The ranges for the mean number of units on each drinking day have been updated from 2016 onwards to provide a more precise measure, and so data is not comparable with previous years.
Ranges and inclusions up to 2014:
- Less than 1 unit – included 0 through 0.49
- 1 or 2 units – included 0.5 through 2.49
- 3 or 4 units – included 2.5 through 4.49
- More than 4 units – included 4.5 and above
Ranges and inclusions from 2016:
- Less than 1 unit – as described
- 1 unit, to less than 3 units – as described
- 3 units, to less than 5 units – as described
- 5 units or more – as described
Increase in drug prevalence (2016)
The following should be taken into account when looking at changes over time for the drug prevalence measures in part 8; ever taken drugs, taken drugs in the last year and taken drugs in the last month (tables 8.1 to 8.8):
1. Questions on psychoactive substances, which include new psychoactive substances (NPS), previously known as legal highs, and Nitrous Oxide (laughing gas), were included in the calculation of the overall prevalence of drug use measures (ever used, used in last year, used in last month) from 2016. Both are covered by the Psychoactive Substances Act 2016 which restricts the production, sale and supply of such substances.
When psychoactive substances are removed from the 2016 measure, the overall drug prevalence figure falls by 3 percentage points (24.3% to 21.3%). This adjusted version is included as an extra measure in the time series data shown in tables 8.6 to 8.8.
2. In 2016, even when accounting for the addition of psychoactive substances to the measures, there was a large and unexpected rise in the overall drug use prevalence reported; 14.6% in 2014, to 24.3% in 2016.
Further investigations identified that some of this change had been driven by an increased likelihood since 2016 of pupils who said yes to having heard of individual drug types, then not going on to answer questions on whether they had tried them. The overall drug prevalence measure is derived using the responses from these individual drug types (see Appendix C), and so this results in a greater proportion of pupils being excluded from the denominator, as their drug use was considered to be unknown. A pupil not providing a response for just one of the 17 drug types asked about can result in them being excluded from the overall prevalence calculation; the proportion with an unknown overall drug use status increased from 8% in 2014, to 21% in 2016 (non-response proportions in subsequent survey years have been around the same magnitude; 20% in 2018, and 22% in 2021).
Cross checking with a further summary question, pupils’ are asked whether they had ever tried any drug, indicates that most of these pupils had not tried any drugs. Thus, the overall impact of having removed these pupils from the indicator would likely be to increase the prevalence rates.
Neither the reason for this, nor how much of the change in prevalence between 2014 and 2016 this accounts for, is clear. However, some level of genuine increase was still apparent. If the overall drug use prevalence figure were to be adjusted based on the response to the summary drug use question, then the estimated prevalence for 2016 would be 21.5%. However, due to the amount of uncertainty in deriving this figure, it has not been presented in the publication.
This also affects prevalence measures for individual drug types, though to a lesser extent. This is because a pupil not answering a question for one drug type (and so being excluded from the overall prevalence calculation), will not impact their inclusion for other drug types about which they did provide a response.
The changes in the proportions of pupils not answering questions for each individual drug type in 2014, 2016 and 2018 are shown below (proportions of non-responses in 2021 were of a similar magnitude to 2016 and 2018):
| Table 1 – Proportion of pupils with non-responses to questions about drug use, by drug type |
| 2014 to 2018 |
|
|
|
| |
Percent (%) |
| Variable (drug type) |
2014 |
2016 |
2018 |
| If used amphetamines |
3.3 |
3.8 |
3.7 |
| If used cannabis |
2.7 |
3.3 |
3.7 |
| If used cocaine |
2.7 |
5.1 |
4.5 |
| If used crack |
2.5 |
4.5 |
4.0 |
| If used ecstasy |
2.6 |
3.9 |
4.2 |
If used glue, gas, aerosols or solvents
(volatile substances) |
2.9 |
5.2 |
4.8 |
| If used heroin |
2.3 |
4.6 |
4.4 |
| If used ketamine |
2.2 |
3.8 |
3.5 |
If used new psychoactive substances
(previously known as legal highs) |
- |
4.2 |
4.3 |
| If used LSD |
2.2 |
3.6 |
3.4 |
| If used mephedrone |
2.3 |
4.1 |
3.7 |
| If used magic mushrooms |
2.6 |
4.0 |
3.6 |
| If used methadone |
2.6 |
4.1 |
3.9 |
| If used nitrous oxide |
- |
5.1 |
4.7 |
| If used other drugs |
2.8 |
5.3 |
4.9 |
| If used poppers |
2.5 |
3.4 |
3.3 |
| If used tranquilisers |
2.1 |
3.6 |
3.8 |
| If used any drugs (derived from all above) |
7.8 |
21.2 |
19.7 |
If used any Class A drugs (derived from
class A drug variables above) |
6.5 |
15.8 |
14.5 |
Other drug time series data in the report is not affected as it is derived from different questions e.g. usual frequency of drug use. The affected tables have been footnoted.
Calculation of standard errors and confidence intervals for means (2021)
In 2021 the tool for statistical processing was changed from SAS to R Studio. As part of this update there was a minor correction made to the method of calculation for standard errors and confidence intervals of means where sub-populations are involved (e.g. mean units of alcohol consumed last week, by age). The method used in SAS creates a separate analysis of the sub-populations, where you treat the sample sizes in each subpopulation as fixed, and you perform analysis within each group independently. The method used in R uses a statistically valid sub-population analysis instead, where the total number of units in the sub-populations is not known with certainty. This resulted in very minor differences in the outputs. For standard errors of the means, in most cases the difference was less than 0.01 and never more than 0.1. For confidence intervals of the means, in most cases the difference was less than 0.02 and never more than 0.2. The mean calculations themselves were not affected.
The affected data is from tables: 1.8 and 1.9 (standard errors for mean cigarette consumption), 5.8 and 5.9 (standard errors for mean number of drinking days), 5.12b, 5.18b and 5.19 (standard errors for mean units of alcohol consumption), and A19 (confidence intervals for mean units of alcohol consumption).
Survey mode - school versus home based
The mode used to collect survey data on smoking, drinking and use of illicit drugs can affect how pupils may answer the questions. For example they may be more willing to admit to some of these behaviours in surveys conducted away from the pupils home. Previous analysis has shown SDD to provide the most accurate measures of undertaking in risky behaviours as it is conducted away from the home environment. More information is available in section A8 of Health and Wellbeing of 15-year-olds in England - Main findings from the What About YOUth? Survey 2014.
There is information on comparisons with other sources at the end of parts 1, 5 and 8.
Survey mode - interviewer versus teacher led 2021
The SDD survey is normally conducted in schools by an Ipsos interviewer under exam conditions. However in 2021 due to the Covid pandemic, schools were also offered the option to complete themselves. Guidance was given to schools for how to run the survey and this emulated the interviewer led survey. The data tables named 'Data quality tables - Mode of survey delivery' show comparisons for key estimates when collected via the interviewer led or school led option.
There was a trend for pupils, in particular girls, to declare some more risky behaviours when the school led the survey themselves rather than an external interviewer. In response to smoking and vaping questions girls were more likely to say they had ever smoked or were current smokers, or had ever used or were current e-cigarette users, when the survey was school led rather than interviewer led. This was not apparent for boys. Girls and boys were both more likely to say that they had ever drank alcohol when the school led the survey rather than interviewer led. For the analysis, school led and interviewer led responses have been combined and presented together. As discussed above, we are aware that where the survey is conducted can influence response, and SDD has been shown to provide the more accurate measure as collected away from the home setting. These results also show that how the survey is conducted within the school may also influence pupil response.
