Questionnaire design and development

This section describes the mode, structure and content of the HSE FS stage one questionnaires. It also describes any substantial differences between the content of the Health Survey for England Feasibility Survey (HSE FS) and the Health Survey for England (HSE) face-to-face survey.

The stage one survey adopted a design with ‘mobile first’ completion in mind, in order to increase levels of engagement. Evidence from push-to-web surveys [3] suggests that participants are more likely to complete surveys using mobile devices, such as smartphones or tablets, rather than desktop or laptop computers. [1] ‘Mobile first’ principles state that question design should limit the cognitive burden on participants and the manual effort required in completing the survey. The main face-to-face HSE questionnaire is complex, collecting health data using lengthy questions and instructions which require cognitive effort and engagement. Maintaining comparability with the face-to-face survey meant that adjustments to question wording were not possible and as such it is difficult to determine what impact this may have had on participants completing the HSE FS questionnaire.

To ensure that the online questionnaire adhered to ‘mobile first’ principles, the following design conventions were followed.

• using language appropriate across devices for example “select” rather than “click”
• displaying only one question per screen, to minimise the need for scrolling on mobile devices or tablets
• avoiding conversational phrases (for example “May we ask you...”, “Now thinking about…”) that would have been used in the face-to-face interview
• “Don’t know” and “prefer not to say” options were not shown initially on screen as they can increase item non-response due to the absence of an interviewer to motivate the participant to provide full and accurate answers. Instead, if a participant tried to proceed without answering a question, they were presented with the additional two options at the bottom of the response list

However, other areas considered as good practice for ‘mobile first’ surveys could not be incorporated in the HSE FS because of the need for the survey measures to be the same as in the face-to-face survey. These included the two areas listed below. 

• Keeping the wording of the question stems and answer options brief and using simple language; limiting the number of answer options presented on screen. Question wording and answer options could not be changed. 
• Avoiding the use of open-text and open-numeric question formats so to minimise the need to use a device keyboard. This was not possible for certain topic areas such as general health where up six verbatim responses for longstanding conditions or illnesses were recorded, nor within the smoking module where the number of cigarettes smoked during the week and weekend were collated.   

Many of the best practice guidelines for the online survey also applied to the paper questionnaire. The following points also applied.

• Unlike the online survey, the paper questionnaire did not include “don’t know” and “prefer not to say” options as they could increase item non-response. It is expected that people would skip the question if they did not want to answer it.
• As the paper questionnaire needed to be identical in question wording to the online survey, the resulting size of the booklet gave cause for concern with regards to participant perception of burden and hence potential non-response. Whilst the content could not be reduced, the layout was redesigned to reduce the number of pages, ensuring that visual attractiveness was retained through minimising the use of ‘white space’.
• It is important that routing is as clear and simple as possible, as participants must navigate the routing themselves within a paper self-completion. It was not possible to reduce the complexity of the routing but instead, questions that relied heavily on routing were clearly written and visual aids such as arrows and bold text were incorporated into the questions.
• The presentation of some questions was amended slightly on the paper questionnaire to make it easier for participants to complete. For example, a grid format was adopted where appropriate, combining questions that would normally be asked separately in the face-to-face survey. For example, questions about the heaviest drinking day, type of alcohol and quantities of alcohol consumed were presented in a grid format with options to select the type of drink and amount consumed.

The stage one survey included many of the core topics covered in the latest published HSE face-to-face survey, i.e. Health Survey for England (HSE) 2019. To ensure comparability with the Health Survey for England (HSE) face-to-face survey, no substantial adaptations were made to the wording of questions, except for the demographic variables listed below. A copy of the questionnaire and participant material is available on request from the Surveys Team by emailing [email protected].​​​

• In the HSE face-to-face survey, sex is interviewer coded (male or female). In the  Health Survey for England Feasibility Survey (HSE FS), participants were presented with additional categories of ‘other’ and ‘prefer not to say’ [4]
• The HSE face-to-face survey collects household income using a show card containing fifteen income response bands for weekly, monthly and annual amounts. It was not possible to replicate this in the self-completion modes. Instead, a simpler income question was used: 10 response bands reflecting weekly/annual income [5]
• In the HSE face-to-face survey, detailed industry (SIC) and occupation (SOC) information is collected by asking several questions about the participant’s working status and the type of job that they do. From this, other socio-economic classifications, such as NS-SEC can be derived. Collecting SOC information is time-consuming and requires interviewers to probe for more detail. If standard SOC questions are included in a self-completion questionnaire, there is a high chance of item non-response. It is also likely that those who do complete the questions will not give enough information to allow subsequent coding. Further, SIC and SOC require expert coding, which is costly. As such, for the FS a self-coded five-class [6] version of NS-SEC was derived from a combination of information on occupation (self-classified into eight categories – see Appendix A for questionnaire wording); and an employment status variable that captured information on employment status and size of the organisation.

The topics covered in the HSE FS, including any differences between the online and paper questionnaire used on the HSE FS, are summarised in Figure 1.

Figure 1: Summary of topic areas covered in the stage one adult survey

Development of the online and paper questionnaires

Parents of children aged 0 to 12 were invited to complete a short paper questionnaire about their child. As appropriate, an additional paper questionnaire for children aged 8 to 12 to complete was enclosed. Children aged 13 to 15 could either complete the survey online or as a paper questionnaire.

The questions for the stage two child survey were all taken from the paper self-completion section of the HSE 2019 face-to-face survey, except for the demographic questions which were taken from other push-to-web surveys. [8] Hence, very little development was required for the HSE FS paper questionnaire. The online survey required some development to incorporate routing and ‘don’t know’ and ‘prefer not to say’ to the answer options.

The content covered in the stage two questionnaires differed depending on the age of the child. Figure 2 outlines the content covered in each questionnaire.

Figure 2: Content of the stage two questionnaires by age group