Skip to main content

Check for National Data Opt-outs Service

This service is the technical solution we've provided, to help you comply with national data opt-out policy. 

Use the secure Message Exchange for Social Care and Health (MESH) to send lists of NHS numbers to be checked against the national data opt-out repository on the Spine. The service will remove those with opt-outs. You will get back a list of NHS numbers for the records you can use or disclose for planning or research, so you can comply with the policy.

Top tasks

Patients should visit nhs.uk/your-nhs-data-matters to set their opt-out choice.

See compliance with the national data opt-out for full details of what you need to do to become compliant, by March 2020.

This guidance is for those working in organisations that use or disclose data that falls within the scope of national data opt-out policy. The Check for National Data Opt-outs Service is a technical solution that health and care organisations can use to check for opt-outs before using or disclosing data.

The four principal GP IT system suppliers are implementing Check for National Data Opt-outs functionality within their clinical systems, so GP practices only need to install the MESH service separately if they disclose data outside their clinical systems. 

Set up MESH

MESH is NHS Digital's secure messaging service. It supports two-way transfer of data between NHS Digital and health and care organisations. Data is fully encrypted during transit. The service uses MESH to allow users to send a data file containing a list of patient NHS numbers to be checked against the Spine repository, where information about national data opt-outs is stored. The service then uses MESH to send back an updated version of the data file with the NHS numbers of those who have opted out removed.

Diagram shows process for transfer of lists of NHS numbers to be checked, between health and care organisations and the Spine interface, as described above on the page.

MESH can be accessed using the MESH client, the most widely-used option, or through systems that have been integrated using the MESH API. This guidance tells you how to set up and configure the MESH client, and how to use the service to check for national data opt-outs.

In most cases systems suppliers or IT administrators will handle the set-up of MESH for each organisation, and end users should get in touch with their organisation's IT contacts with any questions about setting up the service, including installation, access rights or running the MESH client.

Existing MESH users

If your organisation already uses a MESH client, you may be able to use your existing client for this service. 

You can also choose whether to use the existing mailbox, or request a new one. To help you decide, consider:

  • who has access to the files in that mailbox, and if it is appropriate for the users to access both lists of NHS numbers, and other data being transferred for other reasons
  • how it is set up, including automatic transfers or deletions already in place that might affect workflow

1. Request a new MESH mailbox

If you're installing MESH for the first time, you'll be prompted to request a MESH mailbox and set it up as part of the technical instructions. If you are going to use an existing MESH client, you can still decide to request a new mailbox specifically for the Check for National Data Opt-outs Service.

The service level agreement for the creation of a new MESH mailbox is 10 working days. When your request is completed, you'll be sent a mailbox name and authentication key to add to your MESH installation.

How to fill in the MESH mailbox application form

Most fields are self-explanatory but there are specific answers to include in the form for Check for National Data Opt-outs Service mailbox requests.

Under section A, in the fields:

  • 'Name of clinical system supplier' - enter 'Check for National Data Opt-outs Service'
  • 'Email address of clinical system supplier' - provide contact details for your technical lead or support service

Under section B, in the fields:

  • 'Workflow to be supported by the mailbox' - enter workflow ID SPINE_NTT_UPHOLDING
  • 'Summary of requirement' - enter 'This is an existing workflow to allow NHS numbers to be checked for national data opt-outs through the Check for National Data Opt-outs service'

Under section C, in the fields:

  • 'Message format' - enter .DAT
  • 'Individual message size' - enter 'Other: varies according to specific data use/dissemination'
  • 'Message delivery time' - choose whether you are most likely to send messages within or outside business hours. You won't be restricted to your choice.
  • 'Message transmission frequency' - enter how often you think you might use the service, for example weekly or monthly

2. Install the MESH Client (new users only)

If your organisation already has the MESH client installed, you may be able to reuse this instead of installing a new instance. 

To install a new MESH client, follow the instructions on the MESH technical information page. Full guidance and downloads are given in the installation pack.

3. Set up an additional MESH mailbox (existing users only)

If you decide to use a new mailbox with an existing client, you need to edit the MESH configuration file, meshclient.cfg, which will be in the directory and folder where the MESH client has been installed.

Add a new client section by including the following code in the configuration file:

COPY
<Client>

<ClientIdentity>mailbox name</ClientIdentity>

<ClientAuthentication>authentication key</ClientAuthentication>

<MailboxType>MESH</MailboxType>

<InterfaceRoot>C:\MESH-DATA-HOME\Mailbox2</InterfaceRoot>

<CollectReport>Y</CollectReport>

<TransferReport>Y</TransferReport>

<PollReport>N</PollReport>

<SaveSent>Y</SaveSent>

</Client>

HTML

Enter the mailbox name and authentication key you were sent when your MESH mailbox was set up. Make sure the location in the InterfaceRoot field matches the directory location where the MESH client has been installed, and ensure a corresponding folder is created in the root directory location.

See an example of a multi mailbox configuration file

Use the service: guidance on how to prepare, send and receive your files.

Once you have configured MESH, you can use it to send lists of NHS numbers to be checked, and receive amended lists back.

Before using the service, please review the Check for National Data Opt-outs licence agreement, which notes the rights and conditions upon which your organisation may use the Check for National Data Opt-outs service provided by NHS Digital

Creating data and control files

To send a set of NHS numbers to the Check for National Data Opt-outs service via MESH, you need to create two paired files. These are:

  • .dat file – the data file containing the list of NHS numbers that need to be checked
  • .ctl file – the control file that contains the information required to identify, transmit and audit the data file

The paired files must have exactly the same name, apart from their respective extensions.

You can use any text editor, including Notepad or Wordpad, to create or edit the files.

Outbound data file (.dat)

This file must only contain a list of NHS numbers, each followed by a comma and  a carriage return (including the final one). Including any other details will stop the file being processed. Files created with a .csv extension type can be renamed to a .dat file. 

See an example of a data file containing dummy NHS Numbers 

Outbound control file (.ctl)

This file tells the MESH server to transfer the .dat file to the Check for National Data Opt-outs service.

SPINE_NTT_UPHOLDING is the workflow identifier for this service. You must copy this exactly to the workflow identifier (WorkflowId) tag, as shown in the example .ctl file below.

The control file must match the format of the following example, with the data items replaced with your own organisation’s details. Without these changes your file will not be successfully delivered and processed.
 

COPY
<DTSControl>

<Version>1.0</Version>

<AddressType>DTS</AddressType>

<MessageType>Data</MessageType>            

<WorkflowId>SPINE_NTT_UPHOLDING</WorkflowId>

<To_DTS>X26HC036</To_DTS>

<From_DTS>XXXXXXX</From_DTS> 

<Subject>NDOP Response</Subject>

 <LocalId>EXAMPLENDOP-14122018-0001</LocalId> 

<Compress>Y</Compress>

<AllowChunking>Y</AllowChunking>

<Encrypted>N</Encrypted>

</DTSControl>

HTML

The FromDTS field should contain your organisation's mailbox ID. You should provide a value for the Local Identifier (LocalId) tag in the .ctl file(s) particularly if you're sending lots of different files. Files returned by the service will contain the same LocalId so that you can match returned files with the original source files.

The code <AllowChunking>Y</AllowChunking> tells MESH to allow large files to be processed in 20Mb chunks. Without this configuration value in the .ctl file, a file larger than 100 Mb will make the transfer fail.

Naming the data and control files

Data files submitted to MESH must have unique filenames, so that they can be tracked if there are any issues.

A suggested format to follow is to use your mailbox ID followed by a hyphen then day, month and year of the date the file is produced followed by a hyphen and then a 4-digit sequence number to show what order that file has been produced in that month.

So, for example for an organisation with MESH mailbox id YCLTA53 that had produced a file on 7 February 2019 which was the fifth file produced that month, the file names would be:

  • YCLTA53-070219-0005.dat
  • YCLTA53-070219-0005.ctl

Sending files to the checking service with MESH

When the MESH client is installed the following default file system directory/folder structure is created. The location of the directory will be dependent upon how the MESH client has been installed at your location.

<MESH-DATA-HOME>

\FAILED

\IN

\OUT

\SENT

\TEMP

\TEMP\RCV

Place your correctly named and configured files together in your MESH OUT folder (put the data file in first if moving them individually).

If the MESH client has not been set to automatically run (if it's not in the start-up services enabled on your computer) then after placing the files in the OUT folder you will need to run the MESH Client.

Timescales

How long it will take to get the processed file back depends on how often the MESH client is configured to connect to the mailbox - this is referred to as 'polling'. By default, this is set up to be every 30 minutes, but you can change this in the meshclient.cfg file. Changing the PollPeriod value will change how often the MESH client connects, so <PollPeriod>40</PollPeriod> will result in the MESH client checking every 40 minutes.

The Check for National Data Opt-outs Service is also set to ‘poll’ every 30 minutes to collect new files to be checked and to provide updated files back to the MESH client. This can mean that if the polling times have not been changed it could take up to 1 hour to get a processed file back after you send it. However, we don't recommend shortening the polling time unless you use the service more than once a day.

We're reviewing the polling period for the checking service, and we expect to reduce it to 1 minute during 2019.

Housekeeping

Once data files are sent via MESH, both the control and data files are automatically copied into the SENT folder. It is the responsibility of your organisation to manage the SENT folder and provide the necessary housekeeping to move, delete or archive these files as required.

Check for failed transfers

If a control file has been completed incorrectly it will be moved to the FAILED folder along with the respective data file. You should check this folder periodically for files that could not be sent via MESH.

Status of files sent to the checking service through MESH

Unlike some other services which use MESH to submit files, you will not receive a delivery status update file from MESH - just a processed data file.

If you need to check on the delivery status of any files submitted via MESH, you can either:

Receiving files from the checking service through MESH

Files sent back from the service will be placed in the IN folder, along with an updated control file.

Successful transactions

Provided the service has been able to process the file successfully, it will send back a data file with an updated list of NHS numbers - with any that had a national data opt-out recorded removed from the list. This will be placed in the IN folder along with an updated control file. The control file will have a blank subject field, but will include a status record which confirms the transfer of the file back to the organisation, for example:
 

COPY
<DTSControl>

<Version>1.0</Version>

<AddressType>DTS</AddressType>

<MessageType>Data</MessageType>            

<WorkflowId>SPINE_NTT_UPHOLDING</WorkflowId>

<To_DTS>XXXXXXX</To_DTS> *Your mailbox ID

<From_DTS>X26HC036</From_DTS>

<Subject>(blank)</Subject> *blank field, no subject line

<LocalId>EXAMPLENDOP-14122018-0001</LocalId> *LocalID provided in outbound ctl file

<StatusRecord> *Information on the status of the data transfer

   <DateTime>20180928094223</DateTime>

   <Event> TRANSFER </Event>

   <Status> SUCCESS </Status>

   <StatusCode>00</StatusCode>

   <Description> Transferred to recipient mailbox </Description>

</StatusRecord>

</DTSControl>

HTML

Error messages

If there was something wrong with the file you submitted, you may get a .dat file sent back containing an error message. Some common ones are listed here, along with what to do when you resubmit the file to try again. If these error messages persist, or you get one that you don't understand, contact the National Service Desk via the weblog or call them on 0300 3035035.

FileHasNoContent

Make sure the data file you provided contains only 10-digit NHS Numbers, separated by a ‘,’.

The file has more than one column

Make sure the data file you provided contains only a single column of 10-digit NHS Numbers, separated by a ‘,’.

The file is missing one or more delimiters

Make sure the data file you provided is comma delimited - each NHS number should be followed by a ',' including the last one on the list. See the example .dat file.

The file contains one or more invalid NHS numbers

Make sure the NHS numbers provided are all 10-digits and just consist of numbers.

Failing to receive a file back

If you don't get a file back, check the file has gone from the OUT folder and been copied to the SENT folder. If no:

  • check the MESH client is running
  • check the polling period hasn't been changed in the meshclient.cfg file
  • check your FAILED folder in case the file was not set up correctly and could not be transferred

If yes:

  • check whether any automated processes could be deleting files before you have a chance to collect them

If the problem persists, contact the National Service Desk via the weblog or call them on 0300 3035035.

Test the service

We've created a small set of test NHS numbers you can test with. You should not use dummy NHS numbers from your own systems, as the service will not recognise them.

Follow the process above with these numbers, to test your processes for preparing, sending and receiving files. Create a .dat and .ctl form and send them to the service. The returned file should then contain the 11 NHS numbers which have no opt-out registered.

There are 15 test NHS numbers available, 4 of which have a national data opt-out registered. The test NHS numbers are shown below, together with some of the associated demographic details and the opt-out status:

NHS_No

DoB

Age

GENDER

FAMILY_NAME

TITLE

Opt-out Registered

9990536627

20050203

14

Female

XXTESTPATIENT-TEXR

MISS

No

9990537313

20030503

16

Female

XXTESTPATIENT-TFAD

MISS

No

9990537364

20010518

18

Female

XXTESTPATIENT-TFAH

MISS

No

9990569479

19760518

43

Female

XXTESTPATIENT-TJIN

MRS

Yes

9990569401

19750721

44

Female

XXTESTPATIENT-TJIH

MS

Yes

9990539685

20010627

18

Male

XXTESTPATIENT-TFIK

MR

Yes

9990539642

20000917

19

Male

XXTESTPATIENT-TFIG

MR

Yes

9990539596

19950125

24

Male

XXTESTPATIENT-TFIC

MR

No

9990539502

19831220

36

Male

XXTESTPATIENT-TFHU

MR

No

9990539537

19731203

46

Male

XXTESTPATIENT-TFHX

MR

No

9990539693

19661027

53

Male

XXTESTPATIENT-TFIL

MR

No

9990539634

19550608

64

Male

XXTESTPATIENT-TFIF

MR

No

9990539553

19540324

65

Male

XXTESTPATIENT-TFHZ

MR

No

9990539499

19410707

78

Male

XXTESTPATIENT-TFHT

MR

No

9990539510

19360512

83

Male

XXTESTPATIENT-TFHV

MR

No

Download the test dataset

Other demographic details have been set to the same values for each of the 15 NHS numbers:

Demographic Detail

Setting (for all 15 NHS Numbers)

GIVEN_NAME

DONOTUSE

OTHER_GIVEN_NAME

-

ADDRESS_LINE_1

C/O NHS DIGITAL TEST DATA MANAGER

ADDRESS_LINE_2

SOLUTION ASSURANCE 1 TREVELYAN SQ.

ADDRESS_LINE_3

BOAR LANE

ADDRESS_LINE_4

LEEDS

ADDRESS_LINE_5

WEST YORKSHIRE

POST_CODE

LS1 6AE

GPP_CODE

Y90001

Do not enter test data into live patient systems

The test data provided here must only be used as described here. Test data should not be used for any other purpose, or introduced into systems including real patient data.

Risks include:

  • confusion between real patient and test data which could result in real data being recorded against a test record and vice versa
  • test data booked in to real clinical appointment slots, reducing the number of slots available for real patients 
  • inaccurate reporting for statistics or payments.
  • confusion for users switching between live and test environments
Last edited: 26 November 2019 11:39 am