Submissions via Online Consultation Systems in General Practice : Supporting Information

This is an experimental statistics publication.

Experimental statistics are series of statistics that are in the testing phase and not yet fully developed because of issues or features such as poor coverage, poor data quality or because the data is still in  evaluation.

This publication is classed as experimental statistics for several reasons, notably:

• This is a new collection and not all suppliers of Online Consultation Systems are sending data.
• Different system suppliers collect different data items and work is ongoing to ensure a common understanding of data that is used consistently by all suppliers.
• The data collection is new, and we are still evaluating the overall data quality, completeness, and coverage.

Users should be aware of the status and constraints of this data. The limitations of each measure are explained in the Data Quality section below.

NHS England regularly ask for feedback from users of its publications, and we are particularly keen to receive comments and feedback on our experimental statistics, to help with further development.

Provide Feedback

This information is published to understand more about the scale of the patient demand for general practice services that is submitted to practices using Online Consultation Systems, and to learn more about patient use of digital tools to engage with healthcare services.

Patient use of Online Consultation Systems to send requests to general practice is relatively new and there was only limited use of these tools, in relatively low numbers of practices, when the Covid-19 pandemic began in 2020. Early in the pandemic, the urgent need to reduce in-person encounters meant that Online Consultation Systems were more widely commissioned, and their use was more strongly encouraged.

There are around 20 suppliers of Online Consultation Systems, of which ten are currently providing weekly data for this collection on behalf of – and with the consent of – their practices. More information about system suppliers is available in the Data Quality section. We are working to support those system suppliers yet to provide data and will refresh the time series once they are able to do so.

It is important to note that the data items collected, and overall system functionality varies by supplier.

• There is not currently a common set of data items that all suppliers collect and report and where there appears to be common or mappable data items. This means that there is not necessarily a consistent way of capturing and reporting the data. For example, some suppliers offer pre-populated lists from which patients select the appropriate value, some use customisable collection forms and lists, while other systems use free text fields.
• In addition to variation in the functionality of the different systems, there is also variation in how the information from patients is collected and how user-friendly the software is. For example, some Online Consultation Systems collect data using a series of guided questions alongside free-text fields; some systems ask patients to choose a category, such as headache or stomach pain, and guide the user through a series of follow-up questions, and other systems simply collect free text.
• There is not a single and consistent user interface – some systems collect data online via browsers, others via a mobile phone app, while many Online Consultation Systems offer patients the choice of which to use.

This all means that the quality of the data that we receive is extremely variable and that some suppliers cannot provide certain data items. In terms of functionality, some Online Consultation Systems can manage the entire workflow within the software which means these systems can capture the whole exchange and interaction between patient and the practice, including capturing how the practice responded to the submission (e.g. message, telephone call, etc) and whether an appointment was scheduled. However, not all practices that have systems with this functionality will use it, and some may record activity information in other software packages such as their clinical system appointment book or an alternative worklist. Similarly, some Online Consultation Systems provide only partial, or no workflow management functionality, so all details about the practice response and so on are therefore recorded elsewhere.

As a result of the variation in what the disparate systems can collect, and due to the different ways in which individual practices can chose to implement the systems, the early  publications in the series focus on a few key data items where the completeness is good and there is little ambiguity around what is required.

Not all practices that have Online Consultation Systems receive submissions every month. This could be because:

• practices temporarily chose to withdraw system availability
• all attempted submissions were aborted before they were completed and were therefore not received by the practice

It is important to understand that submissions are written by patients who use the Online Consultation Systems to send requests to their practice for clinical or administrative assistance . We do not collect details about the nature of the patient request. Instead, we are given a summary field that classifies the patient’s request as clinical or administrative in nature. However, the determination of the clinical/administrative status of the request is wholly dependent on information provided by the patients, most of whom are not clinically trained, and this may affect the data quality of this information.

Demand for general practice services, and hence the number of submissions received by practices, are affected by a range of factors such as seasonal variation, public holidays, and the number of days in each month. This should be taken into consideration when comparing submission rates between months.

The publication presents a high-level national time series from April 2023 onwards, showing monthly counts of:

• Practices
  • Open practices
  • Practices known to have an Online Consultation System
  • Practices that received one or more patient submissions via their Online Consultation System
• Patients
  • Total registered patients at all open practices
  • Registered patients at practices known to have an Online Consultation System
  • Registered patients at practices that received one or more patient submissions via an Online Consultation System
• Total submissions received via Online Consultation Systems, and classified as:
  • Clinical
  • Administrative
  • Other/unknown type
• Rate of submissions per 1,000 patients registered with practices known to have an Online Consultation System
• Count and average of submissions by weekday and time of submission (available from April 2024 onwards).

There is also a practice-level table for the most recent month which lists:

• All practices that were open at the beginning of the month
  • PCN, sub-ICB, ICB and region
• The name of their Online Consultation Systems supplier(s), or “None/Unknown” if
  • the practice has not agreed to participate in the collection
  • the practice’s system supplier is not yet contributing to the data collection
  • the practice has no Online Consultation System
• The count of patient submissions received via Online Consultation Systems broken down into
  • clinical
  • administrative
  • other/unknown type requests
• The practice’s registered patient count
• The rate of patient submissions received per 1000 registered patients

Two accompanying machine-readable open data file include the practice-level data since April 2023 and breakdowns of submissions by weekday and time of submission at practice level.

System suppliers are allowed to supply retrospective data and updates, which means each publication is a snapshot at the time of publication, and the time series and monthly practice-level figures may be subject to change in future releases.

It is important to note that no patient identifiable or patient-clinical information is collected or available in this release.

Patient usage of Online Consultation Systems

Although the figures include a count of the patients registered at each practice, the publication cannot tell you how many distinct individuals used the practice’s Online Consultation System to make submissions in any given month nor how many of the practice’s patients have accounts to use the Online Consultation System (for those systems where user registration is required). Because no person-identifiable data is collected, it is not possible to distinguish between a single patient making multiple submissions, and multiple individuals each submitting a single request. This means that no assumptions can be made at a patient-level about usage rates, or about the percentage of the registered patient population that is using Online Consultation Systems to access general practice services.

It also does not tell us how many patients have never registered or used Online Consultation Systems, how many patients are aware that the practice has an Online Consultation System nor how widely and effectively this access route is promoted by the practice.

The collection aspires to collect patient demographic information about the individuals using Online Consultation Systems, but good quality data is not yet available. As a result, no assumptions can be made about the individuals using the systems.

Although submissions are classified as clinical or administrative no further details about the reason for the request are available.

Demand for general practice services

The Online Consultation System does not record requests that are received by general practice by other means, for example by telephone, in person, or via other digital services, which means that counts from Online Consultation Systems do not reflect the entirety of demand on general practice.

Online Consultation Systems might be available for patient-use 247, but most can be switched on or off by the practice. This can allow the practice to manage patient demand and to ensure that they are not overwhelmed by requests. If a practice known to have an Online Consultation System has no submissions in a month, it could mean that it chose not to allow patients to use the system during that period. Similarly, if a practice’s count of patient-submissions is notably lower than in other months, this could be because its Online Consultation System was unavailable for patient-use for part of the month and should not be assumed to indicate a reduction in demand as patient requests may have been received in other ways.

Submissions received from Online Consultation Systems are a measure of demand. However, low numbers of submissions cannot be assumed to mean that there is low demand for general practice services, and in some areas, patients may be entirely unaware that they can access general practice services using an Online Consultation System. In addition, systems may be available only during certain periods meaning that patients needing to contact their practices outside of these times will need to use other means. Furthermore, the nature of the patient cohort varies by practice, with differing levels of IT literacy, varying levels of deprivation and different age profiles all of which may affect the patients’ likelihood of interacting with their practice using these digital tools. Again, low counts of submissions do not necessarily mean low demand but could mean that patients are contacting their practices in other ways. Similarly, some practices’ patient list will include high numbers of patients in residential care homes where requests for general practice provision will be managed differently.

In the practice-level data, the supplier for some practices is listed as “None/Unknown”. It is possible that some practices may not have an Online Consultation System, but in most cases, this is because the practice has not agreed to participate in the collection, or because their system supplier is not yet able to provide the data. However, in these cases, we do not know whether the practice has an Online Consultation System nor the extent to which its patients use it to submit requests to the practice. Because practices can implement their Online Consultation Systems and make them available for patient-use in different ways, it is not possible to make any assumptions about the scale and times of system availability, and it is inappropriate to attempt to estimate potential usage levels by patients at practices not yet covered by the collection.

The first publication of this data covered around 75% of practices who are known to have an Online Consultation System.  We are working closely with the remaining system suppliers and will revise the practice-level data when more suppliers are onboarded; the national time series will be updated with each publication.  Each publication should be viewed as a snapshot of systems availability and usage at the time of publication and users should be aware that this is subject to change.

Appointments

A submission by a patient or their representative using an Online Consultation System helps measure demand so cannot be assumed to result in an appointment. It is only when the practice responds that an appointment might take place, but this publication cannot be used to identify that appointments have happened. Furthermore, it is not possible to make any assumptions about the number of appointments that may result from submissions made via Online Consultation Systems. It is important to note that not all online submissions relate to clinical affairs as some are administrative queries such as chasing hospital referrals or requesting a fit note, letter, or prescription.

No assumptions can be made about the number of appointments that may result from clinical requests, and it is also not safe to assume that appointments do not sometimes occur as a result of administrative submissions made using Online Consultation Systems.

From March 2024 methodological changes were implemented which have resulted to changes in the number of submissions in historic data and the number of practices identified as having a system supplier.

Submissions

From March 2024, the way in which submissions are counted has changed. We now accept the latest submission from each supplier for each time period, where previously we found the most complete submission from suppliers for each time period. This change was made based on anecdotal evidence where although some older submissions may have been more complete they were less accurate. The result is a decrease in submissions due to practices not being included for certain months due to the suppliers not including them in their latest submission.

Practices identified as having an Online Consultation system

From March 2024, the way in which we identify if a practice has an online consultation system has been updated. We use data provided by suppliers to identify practices who have a system. We use the installation date to identify when they acquired a system. If this date is null we assume they had a system from April 2023 and therefore they are counted in every month. This has resulted in an increase in these counts every month as previously those without a date were excluded.

Practice-initiated links

Some Online Consultation Systems have the functionality to send their patients a link to use to complete a submission. As this publication aims to capture only patient-initiated demand using these online tools, any submissions that are generated using a practice-initiated link are out of scope and should not be included in the data provided by system suppliers. However, as not all Online Consultation Systems can identify submissions that are in response to practice-initiated links it is not possible to exclude all such submissions from the data set. Therefore, counts of submissions, particularly during 2023, will be slightly too high although it is not possible to quantify the scale of this issue.

The practice-level csv includes all practices that were open at the beginning of the month, regardless of whether the practice is participating in the collection and whether their supplier has been able to submit data.

Data about the Online Consultation Systems and their use is provided by system suppliers on behalf of the practices that have agreed to participate in the collection. The system suppliers provide a weekly summary of which practices they serve, regardless of whether the Online Consultation System was switched on or off during the week. Because this data applies to a week (Monday to Sunday), and since some weeks will span the beginning or end of a month, the month of each Monday is used as a proxy when producing monthly summaries. This information is used to identify each practice’s system supplier, which practices have more than one supplier and which practices have an Online Consultation System but received no patient-submissions during the month.

Some practices’ system supplier is reported as “None/Unknown” which in most cases means we do not know which system supplier they have because the practice has either not agreed to participate in the collection or because their supplier is not yet providing data.

The submissions are created by patients rather than healthcare professionals and the amount and type of information collected by the Online Consultation Systems varies by supplier. This should be taken into consideration when interpreting the figures in this publication.

The publication includes the count of patient submissions via Online Consultation Systems that are received by practices classified as clinical, administrative, or other/unknown type. Some systems have the functionality to identify aborted, incomplete, timed-out, or otherwise unsuccessful submissions which are not received by practices. However, this functionality is not available from all suppliers and since this publication aims to help understand demand, these unsuccessful submissions are not currently included. If all attempted patient-submissions in a month were aborted/unsuccessful for practices where we know they have a supplier, the practice’s count of received submissions will therefore be zero.

Each practice’s registered patient count is included along with a rate of the submissions received per 1,000 registered patients.

Each monthly publication will provide an Excel file containing a high-level time series at national level and practice-level information on the system suppliers, and patient submissions for the most recent month covered by the publication. It is accompanied by two CSV files providing the practice-level figures since April 2023 and submissions by weekday and time of submission at practice level.

This data collection has a range of data quality issues. This is due to variation in what is collected by different Online Consultation Systems and variation in how the systems are implemented in individual practices. It is important to note that each practice manages their Online Consultation Systems in the way they best feel allows them to deliver care to their patients and there is a widespread variation in how the systems are used. It is also important to note that the Online Consultation Systems were not designed to capture data consistently for analytical purposes.

The main known data quality issues are recorded below.

In some areas, ICBs have commissioned Online Consultation Systems for their practices. Where practices procure their own system supplier, they can select from those available on the Digital First Online Consultation and Video Consultation framework (DFOCVC) or choose to use an alternative supplier. More information about suppliers on the framework can be viewed on the buying catalogue here.

A new Digital Pathways Framework, Digital Services for Integrated Care (DISC), will be established during 2024 and will replace DFOCVC. However, some practices may already have chosen, or may later decide to use a supplier that is not on the framework and of which we are unaware. This means that achieving 100% coverage is unlikely.

Since April 2023, NHS England has received data about patient submissions via Online Consultation Systems from the following suppliers:

• Accurx Limited
• Anima
• AT Technology Services Ltd (AT Tech)
• eConsult Health Limited
• EMIS Health
• Klinik Healthcare Solution UK Ltd
• Patchs
• Patients Know Best (PKB)
• Sensely UK Limited
• Substrakt Health Ltd

From March 2025, two additional suppliers started submitting data:

• Evergreen Life (provided historical data back to January 2024)
• SystmOne (TPP) 

However, some suppliers are currently not able submit every week and we are working closely with them to overcome barriers.

There are also several other system suppliers that we are supporting to join the collection, including:

• Engage Health Systems Limited
• Silicon Practice Ltd

All practices that were open at the beginning of the reporting month are listed in the practice-level defined as.

• Open and active on the first of each month
• Have a prescribing setting of '4'. (This is a setting allocated by NHS Business Services Authority, identifying a site as a GP practice)

For a range of reasons, not all submissions that patients begin are completed and received by the practice. However, as not all system suppliers are able to provide information about aborted, or “unsuccessful”, submissions, and because this publication initially seeks to capture information on demand for general practice services, only successful submissions that are received by practices are included.

Submissions are broken down into clinical, administrative, and other/unknown. The determination of whether a submission is clinical in nature may come from the patient or be allocated by the software which means there is variation in how submissions are classified. Some suppliers are not able to provide information on the submission type and in these cases, the submission is classified as other/unknown. These issues affect the utility of the data as it makes it difficult to quantify the scale of clinical demand as opposed to administrative requests.

Data is available only for practices that have agreed to participate in the collection and given their consent for their system supplier to submit data on their behalf. Practices that are yet to sign up are encouraged to do so to meet their contractual obligations and improve the national understanding of patient demand as well as system availability and usage.

Please note that the figures in this publication must not be compared to or combined with the figures from Appointments in General Practice because:

1. The two publications count different things; Appointments in General Practice captures information recorded in practice and PCN appointment books (i.e. appointments), whereas Submissions via Online Consultation Systems in General Practice reports submissions or requests made by patients to GP Practices via online systems. These submissions can be clinical in nature, or for administrative purposes such as requesting fit notes or referrals.​
2. Some, but not all, submissions made via online consultation systems will result in an appointment. In the instances where a submission does result in an appointment this could be included in both publications. We are not currently able to identify when an appointment resulting from a submission via an Online System is recorded in both publications and so the extent of duplication across the publications is unknown. Therefore , we advise against  combining figures from the two publications, as it could result in double-counting.
3. GP Practices are free to manage their practice systems in the way they feel best allows them to provide care to their patients. Both publications extract data from GP Practices, but from different systems. The differences in how these are used may impact how information is recorded and therefore what is extracted. This could result in an unquantifiable amount of duplication between the publications.

We'd love to know what you think of this publication, including how you use it and any ideas for improvement by completing our short survey here.