Skip to main content
Blog

COVID-19 digital readers: Can you see the line?

Rob Banathy, Senior Product Manager, explains how his team developed the lateral flow device (LFD) digital reader service that can detect nearly twice the number of positive COVID-19 cases during early infection compared to the human eye.

When the government rolled out the use of self-conducted and reported lateral flow devices (LFDs) to support mass asymptomatic testing for the public at the start of 2021, a concern was identified around users interpreting the test incorrectly, particularly when the lines on the LFDs are faint.

Three lateral flow devices. One has a visible positive line and the other two do not

In this scenario, asymptomatic users might not be identified and so they might contribute to a chain of transmission. This is particularly a concern where testing is used in situations such as entry to nursing homes, where the impact of missing asymptomatic users would be devastating.

To help mitigate this risk, we explored whether technology could be used to interpret the result of an LFD more accurately by removing the variation in human eyesight and human bias.


Our approach

We conducted a rapid engagement exercise to find suppliers that might have technology to interpret photos taken of LFDs which could meet our strict public health accuracy requirements.

We discovered, however, that there were no production ready, public health assured solutions available in the market.  This meant we had to quickly work with suppliers, and public health and regulatory teams, to design and test a service that could be used safely and effectively by the general population.

Following a proof of concept and technical evaluation stage, we chose a supplier and we developed a pilot service that used artificial intelligence to interpret the test result from a photo submitted by the user. In doing so, the service acts as ‘software as a medical device’, which meant, amongst other things, that we needed to conduct a clinical trial to prove the public health accuracy, safety and effectiveness of the service.

Interpreting LFD results in real time for a general population had never been done anywhere in the world before.

We also had to understand the associated implications of automated decision making in our approach, making sure our service design provided users with an alternative option of reporting their results.  And finally – we had to demonstrate that the service was simple and safe to use.


The challenges

There were several challenges to overcome before we could deliver this service, not least being that interpreting LFD results in real time for a general population had never been done before.

Mindful of inclusivity and accessibility needs, we decided to pursue a web service approach rather than a native app approach, meaning that our goal was to provide a service that any user with an internet connected device with a camera could use and they did not need to download any software or app to use the service. 

Due to the limited photo quality validation that this approach presented, we relied heavily on user research to understand how we could design the service to meet user and public health needs to deliver a simple to use, accurate and scalable web service.

The technical integration challenge was significant, with the need to integrate the service across 3 separate cloud technology stacks, involving numerous interchanges with the technical components, whilst meeting demanding system availability and performance requirements.


How we overcame them

Collaboration and working in the open across teams, organisations and delivery partners was key.  Having a clear mission, with defined goals and measures of success made this easier.

We adapted our user research methods to work remotely and we built prototypes to not only test our understanding of user and public health needs, but also to demonstrate our emerging service vision with our policy and public health colleagues. 

Relationships were quickly built as we needed advice and support from a range of internal and external partners, including regulatory, service design, information governance, security, commercial, policy, public health and scientific teams.


The pilot

In the pilot, we collected over 100,000 images from care home and primary care staff as they reported their LFD results.  We conducted extensive user research with these users, gaining valuable insight on user behaviours and expectations. We worked with public health colleagues to verify our methods and our research data.  We submitted our evidence to the Medicines and Healthcare products Regulatory Agency (MHRA) and sought their authorisation to use this ‘software as a medical device’ service.

Whilst we waited for this, we worked on ensuring the digital service could meet the security, data protection and performance requirements in a production environment.

We received our authorisation from MHRA in April 2021 and launched the LFD digital reader service in June 2021 to a small number of user cohorts.  This helped us to quickly learn about, and respond to, the needs of users prior to scaling up a nationally available LFD reporting service.

The service outperforms self-read LFD reports by detecting nearly twice the number of true positive cases during early infection.

The results

The LFD digital reader service has now been live for 15 months with more than 8 million digitally read results and zero service outages. Since launch, we’ve gained regulatory authorisation to use the digital reader service for all the types of NHS provided LFDs used by the general population for self-reporting their test results.

We conducted a service evaluation using real world data from our production service. From the evaluation, it’s been demonstrated that the service outperforms self-read LFD reports by detecting nearly twice the number of true positive cases during early infection when the viral loads tend to be lower, and it detects over 25% more true positives cases when viral loads are higher. This makes a significant impact in breaking chains of transmissions and in particular protecting the most vulnerable users.

Also from the service evaluation data, we can see that the average time for a user to complete reporting their LFD result using the digital reader service is approximately the same as when a user manually reports their result.

And over 98% of users who choose to use the service to report their results are able to successfully submit an image of their LFD and receive the digitally read result.


What we've achieved

We've had some great feedback on the digital reader service. Tom Fowler of the UK Health Security Agency (UKHSA) said:

This is a great example of innovation and collaborative working across the system between UKHSA and NHS Digital, using new technologies to make a difference to people. It will improve the impact of our national testing programme in protecting those most vulnerable by increasing the accuracy of LFD test results.

We’ve demonstrated how innovative technology can be used safely and effectively in a general population setting to increase the accuracy of interpreting LFD results and thereby improve public health outcomes. Continual access to user research and user behaviour data has guided our service design to ensure our service meets user needs.

Having clear goals was essential and it allowed us to work collaboratively with academics, clinicians, technology and regulatory specialists to deliver an innovative idea from proof of concept to a full production service used by thousands of users every day.

After evaluating the service, the UKHSA recommended that the service is made available to all users who are reporting their NHS provided LFD test results.



Related subjects

In just over a year, the UK’s COVID-19 testing capacity increased from a few thousand to hundreds of thousands of tests a day. This achievement has been crucial to the fight against the virus – and the sinews of the system are digital.

Author

Last edited: 16 November 2022 9:54 am