Weighting

Weights were calculated to reduce non-response bias caused by systematic differences. A four-stage weighting approach was designed for stage one in line with NatCen’s approach to weighting push-to-web surveys. The final weights adjust for:

• differences between participating and non-participating households
• non-response within households  
• the profile of the responding sample population when compared with the known population totals

At each sampled address, there may have been more than one dwelling and/or household. However, there was no control over which household opened the invitation letter (due to the remote data collection), and a random selection of households is very difficult to operationalise without an interviewer. As a result, no household selection took place in multi-household addresses, and the selection of which household took part was left to chance (i.e. whichever household opened the letter).

The weights to adjust for differences in address/household response rates were calculated in two steps.

Step 1: address participation (W₁)

The aim of the address participation weights is to reduce bias caused by systematic differences between the addresses that participated (i.e. for which at least one questionnaire was received) and those that did not.

The probability of receiving at least one response from an address was modelled using the following address-level characteristics:  

• region
• population density
• deprivation
• output area classification and characteristics of lower layer super output area (LSOA) (for example population age and educational distribution)
• information of survey administration (for example wave number)

An address participation weight (W₁) was created. The weight was calculated as the inverse of the probability of response, i.e. W₁ = 1 / P_Address

Step 2: adult non-response weight (W₂)

This stage of the weighting aims to reduce bias caused by systematic differences in the number of completed surveys returned by responding households.

Up to two adults aged 16 and over were eligible to take part in stage one. However, in practice more adults took part in some households (there were three households where not two but three questionnaires were received) [12]. Non-response weights (W₂) were calculated to reduce bias from adult non-response within households with more than one adult. Among households with more than one adult, a model was used to predict the number of adult responses (1 or 2)[13] using household-level and geographic variables as the independent variables, these included:

• harmonised [14] number of adults and children
• household type
• region
• population density, urban/rural
• output area classification and characteristics of lower layer super output area (LSOA)

Participants in single-adult households were not included in the model and were given a non-response weight of one.

Adult combined weight (W₃)

A combined adult weight as the product of the address/household participation weight and the non-response weight for adults was then created, i.e. W₃ = W₁ * W_2.

Adult calibration weight (W₄)

The final stage of the weighting aims to reduce any residual non-response bias at the individual level using calibration weighting. Calibration weighting adjusts the weights so that characteristics of the weighted achieved sample are in-line with the latest Office for National Statistics (ONS) mid-year population estimates for sex/age groups and region.

The weighted estimates were compared for household size, ethnicity and harmonised housing tenure against the latest Labour Force Survey. Consideration was given to whether these additional variables should be added to the calibration matrix [15] The adult combined weight (W₃) was used as the starting weight for the calibration. The aim of the calibration weighting is to reduce non-response bias resulting from differential non-response at the household and individual level.

Weights are designed to reduce bias but in doing so they also reduce precision. Child weights would need to account for household level non-response at stage one and for household and individual level response at stage two. Due to the small sample sizes achieved for children in each age group (see Table 13), stage two data were not weighted, and analysis is presented on unweighted data.