Skip to main content
Publication

General Practitioner Workforce in Alternative Settings July 2019 - September 2019, Experimental Statistics

This is part of

Experimental statistics
Publication date:
Geographic coverage:
England
Date range:
01 Jul 2019 to 30 Sep 2019

Data Quality

Accuracy and Reliability

CCGs were asked to provide details of commissioned GP hours for the settings specified. Some CCGs were unable to supply details of commissioned hours but instead entered provided (actual) or rota’d hours or an estimate. Some CCGs commented in the free-text field of the template, that commissioned and actual hours may be very different or that they commissioned according to need rather than for a set number of hours.

Use of the flags to identify that the CCG had commissioned a service, provided actual or rota’d hours or submitted estimated figures has given us more information about commissioning activity but increased uncertainty about the reliability of the figures provided.

62 CCGs flagged at least one of their providers as delivering a service in which the GP hours may not necessarily be specified or identifiable. Of these CCGs, 58 nonetheless entered hours for 120 flagged providers and the submitted hours were included in the calculations to estimate FTE GP counts. We recognise that it is possible that these hours, which equate to an estimated 550 FTE GPs (figures from the remaining four CCGs were excluded for data quality purposes or null values) may not be limited to those delivered by GPs but could include all staff delivering patient care. These 550 FTE GPs compare to an estimated 440 FTE GPs from 23 CCGs (and 65 providers) April to June 2019 where comments provided in the free-text field noted that provision is commissioned as a service.  Again, users are urged to interpret all these FTE estimates with care. 

82 CCGs said they had entered actual, rota’d, rostered or a combination of all of these types of hours for one or more provider while 48 estimated figures for one or more service provider. Some CCGs flagged their data against multiple categories further reducing confidence in the figures’ reliability.

CCGs do not necessarily commission services for every setting. Of the five main categories, (Extended Access, out-of-hours, 111/IUC, Type 1/2 GP Streaming and UTC/MIU/Type 3/4 GP Streaming), only six CCGs entered counts against all five while 71 provided figures for only one.

Where a CCG did not provide figures for a particular setting, we do not know whether this is because they do not commission GP hours in these settings or because the data are not available. Because of the very wide variation in the way in which such services are commissioned by CCGs, it is not possible or appropriate to estimate for missing data or to make any assumptions about provision in one area or region based upon the submissions of another.

The FTE estimates in the Results section are therefore based upon a combination of commissioned, actual and estimated figures and should be treated with caution. Furthermore, because the figures provided are estimated full-time equivalent counts derived from this hours data, they should not be used to draw any definitive conclusions about the general practitioner workforce and must not be added to counts published elsewhere to make assumptions about the capacity of general practices within the primary care sector.

Data Coverage

The GPFV Monitoring Survey is a mandatory collection and all CCGs are expected to respond.

Of the 191 CCGs, responses were received from 189. Returns from two CCGs failed the validation checks while two submitted a null return for the alternative settings element of the collection. Nine respondents reported that another CCG commissioned on their behalf although this did not necessarily agree with what that named CCG reported in their own submission, and overall there were inconsistencies in the information provided relating to commissioning activity. It is clear that where commissioning takes place jointly or where one CCG commissions on behalf of another, figures were not necessarily reported consistently or, indeed, reported at all which undermines confidence in the reliability of the data. Similarly, because of the inconsistencies in how CCGs described commissioning activity, it is also not possible to be certain that hours have not been double-counted where services are jointly commissioned or organised across several CCGs.

Thirteen CCGs submitted no numerical data and provided no explanation for why this was the case. This may be because the CCGs do not commission any such services, because the data were not available or for other reasons unknown. However, of these thirteen CCGs, eight submitted some hours in the last collection but provided no figures for this return; discrepancies in the hours submitted for some providers between reporting periods mean users are urged to interpret these figures with care.

Data Completeness

There continued to be issues with the quality of the data submitted using the GPFV Monitoring Survey and there remains a risk that in some cases, GP hours could be double counted, see the Coherence and Comparability section for further detail.

For the first publication, we agreed data cleaning measures with our stakeholders which we applied again for this collection for the 165 submitting CCGs. For the first publication, we relied heavily upon the free-text comments fields for additional contextual information that supported data cleaning and validation. In this submission, half the submitted records (52%) were supported by comments in the free-text field as compared to three quarters (77%) in the first collection. It is possible that the introduction of the flags to identify “Service commissioned”, “Actual” or “Estimated” hours may have reduced CCGs’ need to use the free-text fields but it is also possible that important contextual information was not entered  this time and that we included hours that were excluded in light of this contextual detail in the last release.

Note that figures from the same CCG may have been excluded for one or more of the reasons listed.

  • The total number of clinical hours were provided and the proportion that are provided by a GP could not be identified.
    Some of the figures submitted by one CCG were removed.
  • The provider was identified as a hospital or NHS trust. Details of GPs employed by trusts and hospitals should be included in the Electronic Staff Record and published in the NHS Workforce Statistics.
    As we expect that details of these individuals and their working patterns are already captured and reported in other NHS Digital Official Statistics, these figures were excluded to avoid double counting. This affected some figures submitted by 54 CCGs.
    We are advised that it is possible that some GPs working in hospitals or NHS trusts may be employed directly by the CCG but as record-level data were not collected and no information on the contractual arrangements were available, the stakeholder group decided that all such records should be excluded.
  • The provider was identified as a general practice and the provision was recorded as “Other” and / or the comments noted the figures applied to GP with Special Interest (GPwSI). In these cases, we believe the working hours are already captured in the NWRS and reported in the General Practice Workforce Statistics. Records from four CCGs were excluded for this reason.
    • To be consistent, records from two other CCGs were excluded where the provision was specified to be GPwSI

Following the data cleaning activity:

  • All figures were excluded for four CCGs for one or a combination of the reasons listed
  • A further 51 CCGs had figures for some providers excluded
Breakdown of figures included after data cleansing

Please refer to the previous publication at https://digital.nhs.uk/data-and-information/publications/statistical/general-practitioner-workforce-in-alternative-settings/general-practitioner-workforce-in-alternative-settings for details of the data cleaning activity specific to that release.

We can conclude therefore that the data quality has not improved this quarter despite the changes made to the data collection.

Relevance

Figures were provided at CCG-level but reported at national-level only. Because of the complex and contradictory commissioning arrangements, it will not be possible to produce reliable CCG-level or any other sub-national figures based upon data collected using the GPFV Monitoring Survey.

These figures should be treated with caution. Some services such as Urgent Treatment Centres (UTCs) are still in the process of being set up – in this case with provision transferring from Type 3/4 services – which means that apparent increases or decreases in a particular setting may be subject to misinterpretation.

Timeliness and Punctuality

The online collection was open for submissions between 1 and 30 October 2019 and estimates have been calculated and released as soon as possible after collection.

In line with the previous collection, several CCGs noted that data could not be provided in time. However, as the alternative settings part of the collection was a subset of NHS England and NHS Improvement’s GPFV monitoring survey, there was no scope for extending the collection period.

The decision to use this collection mechanism was jointly taken with the key stakeholders (DHSC, NHS England and NHS Improvement and Health Education England) to ensure that initial data could be collected and released as soon as possible and to reduce collection costs, thereby saving public money.

The joint collection activity was beneficial to the return rate as participation in the GPFV Monitoring Survey is mandatory and responses were therefore expected from all CCGs.

Accessibility and Clarity

Estimates are provided for England only and no data are available at sub-national level for the reasons already outlined.

These are estimates based upon a combination of commissioned, provided, rostered, rota’d or estimated hours which assume that full-time equivalence is 37½ hours per week and that a quarter was a full 13 weeks.

No identifiable person-level data was collected and there was no need for statistical disclosure control to be applied.

We are releasing online in tabular format.

Coherence and Comparability

Unless record-level data can be collected from each provider, collecting figures from CCGs remains the only source of information for the work GPs may do in these settings. However, as the estimated FTE counts are not based upon individual record-level data, no assumptions can be made about the number of individuals (headcount) or about actual GP capacity. Because in many cases, the hours submitted are commissioned hours (which does not necessarily imply that these hours were worked) or estimated values, it is possible that some double-counting or over-reporting of hours has taken place. This issue could be mitigated by collecting record-level data directly from the providers which should enable individuals to be identified and their working hours to be recorded accurately.

As described in the accuracy and reliability sections, these estimates are not comparable with releases based upon record-level data as provided in the GP workforce publication, NHS Workforce or independent sector Official Statistics.

It is possible that GP hours recorded for a particular provider passed validation and were included in the reported figures for the last collection but that additional information, for example in the comments field resulted in counts for that organisation being excluded for data quality purposes in this release. For example, around 86,000 hours (equal to an estimated 176 FTE GPs) were included for April to June 2019 from 36 providers whose hours were either blank or excluded for data quality purposes in this submission. Of these 36 providers, additional information from the CCGs identified six providers as hospitals, resulting in the exclusion of over 17,000 hours (around 35½ FTE), 29 entered no hours data in this return and the remaining CCG commented that the hours entered were total clinical hours commissioned from which GP-specific hours could not be deduced.

It is possible that figures were excluded in the last release, for example when the CCG commented “these are clinical hours and we do not know the GP split”. However, if this additional contextual information was not entered for the same provider in this collection, we assumed that the figures accurately reported the commissioned hours and included them in the calculations.

Unless figures were excluded for the criteria outlined on the Data Completeness section, we expect that the recorded figures were accurate at the time of submission and that any apparent changes in commissioning behaviour are real. However, such apparent changes in commissioning activity from one collection to another may also be due to decreasing data quality.

GP FTE estimates have been calculated based upon figures from 376 named providers in this collection and 374 for the reporting period April to June 2019. These represent a total of 438 named provider organisations as not all providers were listed and passed the data quality validations for both returns.  Of these providers, figures for 304 passed the data quality checks and were included in the estimated FTE counts in April to June and also July to September; figures for the remaining 70 and 72 organisations contributed to the FTE estimates for only April to June and July to October respectively. This means that although both reporting periods saw a similar number of providers’ information excluded for data quality purposes, in some cases, different providers’ data were excluded.

Assessment of User Needs and Perceptions

Comments and feedback are welcomed by email to enquiries@nhsdigital.nhs.uk or PrimaryCareWorkforce@nhs.net, or by telephone 0300 303 5678.

User needs and feedback are taken into consideration during the production process. We worked collaboratively with our stakeholders, and particularly colleagues in NHS England and NHS Improvement, to formulate the data collection and produce guidance for CCGs. As well as providing written material and an email address for queries, to clarify the requirements of this new element of the collection, we ran several webinars for CCGs.

Performance Cost and Respondent Burden

This was part of an existing collection commissioned by NHS England and NHS Improvement which takes place quarterly. The addition of this section constituted minimal additional burden on the data collections team.

To reduce the burden on CCGs, the template was pre-populated with the names and contact email addresses of any providers identified in the last collection and CCGs were asked to provide hours for July to September 2019 or add details of any new providers.

The request for data for three reporting periods was assessed and approved by the Data Standards Assurance Service on behalf of the by the Data Co-ordination Board.

Confidentiality, Transparency and Security

No record-level data relating to individuals was collected. Only aggregate data relating to hours was returned.

The standard NHS Digital data security and confidentiality policies have been applied in the production of these statistics. The data contained in this publication is Official Statistics. The Code of Practice for Statistics is adhered to throughout the publication cycle along with NHS Digital’s Statistical Governance Policy.

All publications are subject to a standard NHS Digital disclosure risk assessment prior to issue and statistical disclosure control was not necessary for this release.

Last edited: 26 November 2019 5:40 pm