Setting up control files

Each submission to DLP via MESH requires a file pairing, comprising of:

• a .dat file - The data/ payload to be transferred, which in the case of DLP is the ANSI (Windows-1252) encoded CSV submission file. It has to be renamed with the .dat extension
• a .ctl file - This is the control file. It contains the information required to identify, transmit and audit the data file. The control file will need to be updated with your specification ID and filename. This is explained in more detail 

Each file pairing will need to share the same exact name, with the .ctl and .dat extension added respectively. 

The control file MUST match the format of the following example, with the items highlighted in bold- replaced with your own organisation’s details, as per the accompanying note (found to the side of the field in question). Without these changes your file will not be successfully delivered and processed. 

<DTSControl>

<Version>1.0</Version> 
 <AddressType>ALL</AddressType> 
 <MessageType>Data</MessageType> 
 <WorkflowId>DSP_EXTRACT</WorkflowId>   
 <From_ESMTP>[email protected]</From_ESMTP> *This needs to be your mailbox ID 
 <To_ESMTP>[email protected]</To_ESMTP> 
 <From_DTS>XXXX</From_DTS> *This needs to be your mailbox ID 
 <To_DTS>X26HC010</To_DTS> 
 <Subject>b2ac587139c34646bc5ca6cbfb600cf5</Subject> *The Specification ID to be used for the submission - you can get this from your data recipient if you are unsure what this is. 

<LocalId>my_submission.csv/LocalId> *The filename for the submission (this MUST be a .csv or .csv.gz file)  <Compress>Y</Compress> 
 <Encrypted>N</Encrypted> 
 <AllowChunking>Y</AllowChunking> 

<DTSControl>

Once the two files are appropriately named and configured, they will need to be placed together in your MESH OUT-folder (.dat first if moved individually).  

After placing in the OUT-folder the user will need to run the MESH Client. This will then send the files to the MESH Server. If the MESH Client is already running the files will be sent when the next MESH Client poll happens (this is configurable within each MESH Client).

Once transferred by MESH, the outbound control file is amended with the transaction details (as per the example below) and placed within the client SENT-folder, along with the paired outbound submission .dat file. 

<DTSControl>

<Version>1.0</Version> 
 <AddressType>ALL</AddressType> 
 <MessageType>Data</MessageType> 
 <From_DTS> XXXX </From_DTS> 
 <To_DTS>X26HC010</To_DTS> 
 <From_ESMTP>XXXX@dts.nhs.uk</From_ESMTP> 
 <To_ESMTP>[email protected]</To_ESMTP> 
 <Subject>b2ac587139c34646bc5ca6cbfb600cf5</Subject>  <LocalId>my_submission.csv</LocalId> 
 <DTSId>20180919102252052139_A39867</DTSId>

<Compress>Y</Compress> 
 <Encrypted>N</Encrypted> 
 <WorkflowId>DSP_EXTRACT</WorkflowId> 
 <IsCompressed>Y</IsCompressed> 
 <AllowChunking>Y</AllowChunking> 
 <StatusRecord>

<DateTime>20180919112341</DateTime> 
 <Event>TRANSFER</Event> 
 <Status>SUCCESS</Status> 
 <StatusCode>00</StatusCode> 
 <Description>Data Transfer success confirmation</Description>

</StatusRecord> </DTSControl> 

This indicates the file has been successfully sent to MESH. Once the outbound message has been received by DLP’s MESH mailbox, DLP will send a response back via MESH, to the sender’s IN-folder, as confirmation of delivery and whether the submission was successful. 

On receipt of the sender’s submission by DLP, the sender’s control file details will be checked to ensure that:

• workflow ID is correct
• specification ID is correct
• Filename is a csv/csv.gz
• senders mesh mailbox is in our ref data

The submission file will be checked to ensure 

• file size is within limits – currently 1GB for csv and 100MB for csv.gz
• the file structure is correct, as per the specification
• data fields conform to any validation rules set in the specification

The outcome of these checks will be sent back to the sender’s IN-folder, again as a .ctl and .dat file pair. The .dat file is the ‘submission_result.json’, which will state whether the submission was successful or not (with the reason given for failure).

Example IN Ctl file for submission response

    <DTSControl>

<Version>1.0</Version>

<AddressType>ALL</AddressType>

<MessageType>Data</MessageType>

<From_DTS>X26HC010</From_DTS>

<To_DTS>XXXX</To_DTS>

<From_ESMTP>[email protected]</From_ESMTP>

<To_ESMTP>XXXX@dts.nhs.uk</To_ESMTP>

<Subject>SubmissionResultDLPv1</Subject>

<LocalId>submission_result.json</LocalId>

<DTSId>20180919102312390939_0B1CFF</DTSId>

<PartnerId></PartnerId>

<Compress>N</Compress>

<Encrypted>N</Encrypted>

<WorkflowId>DSP_EXTRACT</WorkflowId>

<ProcessId></ProcessId>

<DataChecksum></DataChecksum>

<IsCompressed>N</IsCompressed>

<StatusRecord>

<DateTime>20180919112312</DateTime>

<Event>TRANSFER</Event>

<Status>SUCCESS</Status>

<StatusCode>00</StatusCode>

<Description>Transferred to recipient mailbox</Description>

</StatusRecord>

</DTSControl>

Submissions that have successfully passed the first 5 bulleted checks, as listed above, will appear in the DLP User Interface (UI), in the same manner they would had the files been manually uploaded to the system.

Should any validation rules failures be found they will be confirmed in the returned
submission_result.json, indicated by a validation_counts item showing as 1 or more:

Example

"validation_counts": {

"errors": 0, *Validation failures which will result in file rejection – submission failure

"no_action": 5, *Validation failures with no resulting action

"warnings": 1 *Validation failures meriting warning, but not causing submission failure

},

In this case, you will need to login into the DLP UI to download the validation report for your submission, which will fully detail the cause for any failures.

The following case samples display the full range of return results you can expect to receive back, depending upon the success or failure of the submission and the type of errors (if any) found with the file.