Considerations for APMS 2022: Suggested content to remove

In order to make space for new topics, some topics and questions in the 2014 survey will need to be removed. The consultation survey asked respondents to choose their top five priority topics to remove from APMS 2022. The response options consisted of 2014 APMS topics which have had little use or where the prevalence and associations are not expected to change rapidly, and therefore where the 2014 dataset could continue to be valid.

The majority (85%) of survey respondents chose ‘Verbal fluency (animal naming exercise)’, a test of cognition used to detect early signs of dementia, as one of their top priorities to remove (Table 4). In focus group discussions participants were reassured by there also being another more widely used measure of cognitive decline on the survey, the TICS-M.

In the focus groups, participants were hesitant to suggest areas to remove. Participants believed that all topics were valuable, and they did not want to cut topics that could be useful for others. Suggestions included:

• intellectual impairment question could be removed as this mainly picks up on dyslexia in younger age groups, which is not a limit limiting condition.
• sexual behaviour could be reduced as these questions are included in the National Surveys of Sexual Attitudes and Lifestyles (Natsal), which will be in field at the same time as APMS. Equally there were requests by some academic researchers to retain and even extend these questions, and as noted above APMS covers the older population who are excluded from the Natsal sample.
• Menopause was suggested as an area to cut, as it only affects a small number of survey participants. However, those interested in women’s health, were keen to retain these questions.
• items on religion and spirituality, social capital and participation, parenting and childhood neglect, were also suggested as possible topics to reduce rather than cut completely.
• as mentioned above, academics using the alcohol questions suggested the SADQ could be removed

Religion and spirituality’ was selected by two-thirds (66%) of respondents. Other topics that over 100 people chose as a priority to cut included ‘Sexual behaviour’ (46%), ‘Social capital and participation’ (44%), and ‘Menopause’ (43%). The topics from the provided list selected least as priorities to cut included ‘Childhood neglect’ and ‘Caring responsibilities’.

Personality disorder was mentioned four times in the survey in the open response question regarding topics to remove. These respondents suggested shortening this section, removing it as the definitions were too broad, the terms used were stigmatising, and because personality disorders are difficult to assess well in a questionnaire. 

Table 4: Proposal for survey topics to remove 

Source: APMS 2022 Consultation Survey  

The consultation asked respondents which types of outputs they would find helpful and why. Over half (58%) of survey respondents selected short chapter reports as a preferred output (Table 5). Infographics were also a common answer, chosen by 50% of respondents. Over a third (37%) required the overview and detail of a large report. Dashboards, dataset and data tables were fairly evenly split. By contrast, regular users of the APMS who took part in focus groups and events were strongly in favour of a large report.

Table 5: Survey outputs that respondents would find useful 

Source: APMS 2022 Consultation Survey

One large report with data tables

Regular users of APMS warned against replacing the main report with other types of outputs. They favoured one main report with all content in one place. Focus group participants listed the benefits of one report as:

• all information is in one place
• comprehensive but accessible writing, with detailed academic referencing
• the report can be downloaded, printed out, be a reference ‘on the desk’
• the report is easy to navigate, and areas of interest easily searched (for example, using the Ctrl+F function)
• an authoritative, valuable and useful resource
• citable in further publications.

One suggestion to further improve the report was to include an overall prevalence figure to capture ‘any mental disorder’. It was also noted that the chapters on comorbidity were key.

There was some interest in having a dashboard so that people can explore the data themselves. Good examples of these included the PHE Fingertips dashboard. Third sector organisations were particularly interested in such a tool being developed. There was less interest from academic data users, government or the public. Participants were clear that a dashboard should be provided in addition to (and not replace) the full report, data tables and data access.

To further enhance the usefulness of the APMS and potential for secondary analysis projects. Consultation participants made several data-linkage suggestions:

• Mortality data: in order to link to mortality date and reasons why
• NHS Increasing Access to Psychological Therapies (IAPT) data: in order link to therapy access
• government benefits records: to run analysis on Department for Work and Pensions data on welfare benefits and with Her majesty’s Revenue Office (HMRC) data on earnings and income
• use of established Trusted Research Environments and data hubs, such as those at SAIL and ONS

Third sector organisations stated that they would like to make use of secondary analyses that are done and recommend that NHS Digital track what studies are done and raise the profile of where publications can be found. They also recommended producing a synthesis of the secondary analysis findings.

Focus group participants with lived experience had not been aware of the APMS or its outputs prior to taking part in the consultation. They expressed an interest in knowing the survey results. They suggested the survey findings could be shared with those with lived experience via trusted sources for example, Mind social media communications, NHS Mental Health Trust newsletters, NHS website, Royal College of Psychiatry communications. Additionally, they suggested that mainstream media sources could also have a wide reach, such as a BBC documentary about the survey and its findings, or via a podcast. 

The consultation explored whether respondents would recommend potential sample boosts of APMS 2022, and the reasons for this.

Survey respondents were asked to choose their top three priority samples to boost (Table 6). A similar proportion prioritised boosts with younger people (59%), minority ethnic groups (56%) and deprived neighbourhoods (55%). Support was also strong for increasing the overall sample size (45%).

Table 6: Potential sample boosts

Source: APMS 2022 Consultation survey

Focus group participants were divided on whether a young person boost was the best use of boost funding. Those interested in young people’s mental health were keen for this boost. However, participants also noted that the Mental Health of Children and Young People in England is a valuable existing resource which provides information on this age group.    

There was widespread support for a boost of ethnic minority respondents from across government, third sector organisations, data users and the public. The last comprehensive survey of ethnic minority psychiatric illness rates in the community, Ethnic Minority Psychiatric Illness Rates in the Community (EMPIRIC) was conducted in 2000. Participants stressed that this is out of date. A boost of ethnic minority groups would provide a better understanding of how mental health needs have changed over time in these communities.

Furthermore, focus group participants highlight two key reasons for boosting ethnic minority groups

• participants highlighted that boosting the sample would be good for the mental health field. The low number of ethnic minority participants in previous surveys has meant that analysis is often limited to two broad ethnic categories (White and all other minority groups). A more nuanced analysis of findings across different ethnic groups would have the potential to develop stronger conclusions for policy and practice, based on larger sample sizes

• there is evidence of inequalities in mental health diagnosis and treatment by ethnicity. For instance, there are a higher number of black people detained under The Mental Health Act. A boost would help to explore whether this is due to higher prevalence of mental illness or whether other factors should be considered

Participants also put forward considerations for the boosted study design:

• to focus on the largest ethnic minority population groups in England: Black African, Black Caribbean, Bangladeshi, Indian and Pakistani. There was also interest in including Traveller and gypsy groups, however, it was acknowledged that a separate study may work better for this group
• to have additional questions for minority participants questions: for example, migration, generational status, discrimination questions that are multidimensional to explore intersectionality (between ethnicity and gender for example) and mental health-related stigma
• there was also a request for a follow-up qualitative study, which was also a key component of the previous EMPIRIC survey

There was also support for a boost of low-income neighbourhoods. The rationale for boosting this sample included:

• to understand inequalities in mental health needs, diagnosis and access to treatment and support in low income areas compared with other areas.
• to survey households and individuals who are not always reached by national surveys
• to increase the number of people in the sample with low prevalence conditions or who have faced particular adversities, such as experience of having been in prison or face violence

• Department for Work and Pensions (DWP) representatives were particularly supportive of this potential sample boost, as a way of trying to boost benefit claimant numbers. The current sample sizes for claimants, limits utility of the APMS for DWP. Such a boost would support the government’s levelling-up agenda

There was also a request to extend APMS to include Wales. This request was made by those working in the fields of domestic violence or childhood sexual abuse, who also regularly work with the Crime Survey for England and Wales data. This boost would help them to draw conclusions for both countries using the APMS, and alongside the Crime Survey findings.

Over a quarter of participants (28%) highlighted the importance of building qualitative follow-up interviews into the survey design. These could focus on particular subgroups that the survey is uniquely well-placed to identify, such as the qualitative follow-up that took place after the 2007 APMS of people with problem gambling and difficult levels of associated debt.

Suggestions included:

• a qualitative follow-up study about COVID-19 and mental health
• a separate survey or qualitative study of those who do not live in households. Participants acknowledged that household surveys, such as APMS, are unable to include people from all groups who may be at risk of mental health difficulties. They therefore suggested the need to invest in additional studies to assess the mental health of populations living outside of households. This includes those in institutions, hostels, prisons, refuge centres or as well as undocumented populations including refugee and asylum seekers
• a separate study of traveller and gypsy groups and their mental health
• there was also a request to collect more non-response information, to understand whether individuals are not able to take part in APMS due to a health impairment. This would help to understand non-response and whether particular groups are missed and therefore under-estimated in the study