Guide for CAF-aligned DSPT independent assessors

As discussed in the executive summary, one of the key outcomes of this updated guidance documentation is to align the methodologies used by CAF-aligned DSPT independent assessment providers across the sectors. We recognise that each independent assessment provider, will have their own methodology and risk assessment/reporting process.

However, in this document we have outlined a suggested approach, based on industry good practice, that assessment providers should consider throughout their assessment lifecycle.

Similarly to the independent assessment framework, this is not designed to replace existing methodologies and knowledge or experience, particularly where an organisation’s audit and risk committee require audits to be performed and reported in a standard format. However, it acts as a reference point for providers to facilitate and inform alignment across the sector and bolster any gaps in existing methodologies. 

NHS England encourages organisations to choose assured service providers from the NCSC Cyber Resilience Audit Scheme or equivalent. Due to the change in focus and nature of the assessment, it is encouraged that independent assessments are conducted by qualified and skilled assessors who are experienced in, and can competently assess against, the CAF.

There should be 2 outputs of each independent assessment:

1. An assessment of the overall risk associated with the organisation’s security and governance of information control environment. For example, the level of risk associated with controls failing and security and governance of information objectives not being achieved. This will be achieved through testing the individual outcomes providing an assessment of the overall risk position of the organisation.

2. An assessment as to the veracity of the organisation’s self-assessment/CAF-aligned DSPT submission and the independent assessor’s level of confidence that the submission aligns to their assessment of the risk and controls. This will be achieved by testing the self-assessment against the independent assessment. 

In essence, the first output will be an indicator for those outcomes assessed, of any good practice and to the level of risk to the organisation for security and governance of information. The second output will support an internal audit provider in arriving at the assurance level that they are required to provide, and that the organisation is obliged to provide, as per the requirements in outcome A2.b of the CAF-aligned DSPT.

The overall risk evaluation output is seen as key to leading the conversations and improvements required. That is, this updated guidance aims to support:

• enabling NHS organisations to continually improve the quality and consistency of CAF-aligned DSPT submissions across the NHS landscape
• delivering a framework that is adaptable in response to emerging information security, data and health and social care standards
• allowing for a range of bodies to deliver independent assessments in a consistent and easily understood fashion
• helping lead measurable improvement of security and governance of information across the NHS landscape and support annual and incremental improvements in the CAF-aligned DSPT itself
• delivering a framework that better enables and encourages organisations to publish a more granular, evidenced and accurate picture of their organisation’s position in terms of security and governance of information
• delivering a framework that allows for security and governance of information professionals to spend time on-site coaching organisations on security improvement options at the same time as assessing controls and risks
• delivering a framework that helps ensure consistent delivery of ‘independent audit’, internal audit
• enabling and encouraging appropriate feedback and dialogue between NHS England and independent assessors to help inform NHS-wide communications and initiatives to help address common challenges and systemic or thematic security issues and to help inform the development and consumption of NHS England-provided national services around security and governance of information; and
• enabling use of other sources of assurance across the NHS to reduce the burden on organisations and reduce total effort, cost and help minimise duplication of information gathering

Using professional judgement

The CAF aligned DSPT independent assessment guide (including the CAF-aligned DSPT strengthening assurance framework) are not exhaustive. Collectively, these documents will not cover every eventuality and professional judgement will be required in how the standard is met and audited.

The CAF aligned DSPT is outcomes-based, which means an independent assessor will need to use their professional judgment to test if the outcome has been met. This is based on the evidence provided by the organisation and not if the evidence is in line with a prescriptive set of evidence requirements.

Both sets of guidance endeavour to be vendor agnostic. A health and social care organisation may have an excellent vendor-supplied system, which is not referred to in the guides. That is not to discount such a system, which should be implemented and audited on its merits. 

The required standards have to be achievable by those whose digital maturity is 'still developing’. As a consequence, some of the measures outlined could be seen as quite manual or basic in nature. This does not mean that more sophisticated measures cannot be implemented.

When implementing or auditing please pay regard to the intent of the evidence, outcomes and ultimately the 5 objectives of the CAF. It is not the intention of the CAF-aligned DSPT Strengthening Assurance Framework to create tick lists of items to be implemented and audited that do not reflect actual practice.

Understanding risk (risk fundamentals)

Understanding and assessing risk is important throughout the processes of conducting a CAF-aligned DSPT assessment. This section introduces evaluating and quantifying risk, as well as serving as a refresher for those with previous experience in this discipline.

For the purpose of this guide for CAF-aligned DSPT independent assessment providers, the following definition of risk should be used:

'Risk is the effect of uncertainty on objectives.'

This definition of risk can be explained using a combination of 2 key determinants: the likelihood of a certain event occurring (an expression of the ‘uncertainty’ in the definition above) and the impact (or 'effect' in the definition above) such an event would have on the achievement of one or more objectives. Exploring these 2 key determinants further, the guide for CAF-aligned DSPT independent assessment providers defines likelihood as:

'The chance that weaknesses in a set of controls that make up an evidence text, results in a cyber security or information governance incident or breach.'

The definition of impact is:

'Impact is the magnitude of harm to an organisation that could result from a successful threat or breach occurring.'

The risk rating is determined at evidence text item level and comprises 2 elements, likelihood and impact.

The guide for CAF-aligned DSPT independent assessment providers is designed to assess security and governance of information risk, which is defined as:

'Security and governance of information risk is the risk to the organisation’s achievement of its objective of preserving confidentiality, integrity and availability of data assets which support the essential functions.'

To allow the CAF-aligned DSPT independent assessment provider to assess overall security and governance of information risk, the risk equation is expanded to cover 3 important tasks that align to the CAF-aligned DSPT independent assessment workflow.

This means that the CAF-aligned DSPT independent assessment methodology has been designed in a task-by-task format to provide independent assessors the guidance they require to assess likelihood, impact and the final risk rating.

In the first instance, health and care organisations and independent assessment providers should seek to resolve any issues, complaints and disputes in relation to the independent assessment directly. Both organisations would be expected to fully co-operate to resolve the problem.  

Health and social care organisations are responsible for managing any potential or actual conflicts of interests. Interests should be considered prior to contracting or agreeing the independent assessor. 

If issues, complaints or disputes cannot be resolved directly, you can get advice from NHS England via the DSPT Contact us page.

There are 5 core tasks and a number of sub-tasks central to the delivery of all CAF-aligned DSPT independent assessments. This section outlines an overview of the key activities and expected outcomes required for each task:

Task 1: Pre assessment preparation and information gathering    

Activities

Confirm the independent assessment provider, commence initial engagement with health and social care organisation and agree scoping meeting date.

Review CAF-aligned DSPT evidence including previous internal audits and assessments.

Outcomes 

Engaged health and social care organisation sponsor and stakeholder.

Engaged DSPT independent assessment providers.

Task 2: Scoping the assessment

Activities

Conduct detailed scoping meeting to draft and agree terms of reference with input from stakeholders, and sign off from sponsor.

Understand any issues, incidents or changes since last DSPT independent assessment, and review the health and social care organisation’s action plan.

Address any questions the organisation may have and ensure follow-up actions are included in the scope of the independent assessment.

Agree with the organisation the level of access required, timings and availability of staff to assist in the independent assessment. 

Outcomes

Staff knowledge transfer - where a previous independent assessment was completed by another assessment provider.

Finalise and receive formal sign off for DSPT independent assessment terms of reference. See Appendix B for an example template.

Task 3: Performing the assessment

Activities

Formally kick off independent assessment review with health and social care organisation.

Review health and social care organisation's data security and protection documentation using evidence provided through the DSPT. This includes:

• performing interviews and conducting sample testing
• documenting observations and issues as they are identified throughout the DSPT review
• performing risk and confidence evaluations

Discuss with organisation their responses, practices and controls and address outstanding requests to ensure completeness of testing.

Outcomes 

Begin drafting DSPT working papers.

Quantified measure of risk and confidence.

Task 4: Post assessment review and reporting

Activities  

Finalise observations, issues and recommendations including the risk rating of issues. 

Prepare the final report and associated working papers.

Complete a closing out meeting with the health and social care organisation to discuss issues and recommendations.

Support Issue tracking and follow-up work. 

Outcomes 

Draft assessment report including observations, issues and recommendations.

Finalise CAF-aligned DSPT assessment report. See Appendix C for an example template.

Informed discussion on how to improve data security and protection controls.

Task 5: Assessment finalisation and quality management

Activities 

Co-ordinate any further discussion required with health and social care organisation.

Ensure agreed recommendations and health and social care actions (in response to recommendations) are tracked and monitored.

Encourage shadowing, training and knowledge transfer to enable staff to understand the assessment process consistently and effectively. Assess the skills gap in the independent assessment organisation’s staff and support a plan to bridge this gap through a training needs analysis for independent assessors.

Outcomes    

Independent assessment providers to maintain their own audit trail and files.

Training needs analysis and plan to upskill independent assessment staff.

Updated issues and recommendations.

Task 1: Pre-assessment preparation and information gathering

Confirm the independent assessment provider, commence initial engagement with health and social care organisation and agree scoping meeting date 

Before a CAF-aligned DSPT independent assessment can be conducted, pre-work should be undertaken. This primarily involves the health and social care organisation engaging DSPT independent assessment providers, confirming the provider appointed, and agreeing initial meetings between the health and social care organisation and the independent assessment providers. The initial meetings are required to agree scope, confirm stakeholders and sponsors.

Review CAF-aligned DSPT evidence including previous internal audits and assessments

Before the initial engagement, it can be helpful to review previous evidence which may support the CAF-aligned DSPT independent assessment. This could include internal audits and assessments, or other evidence and may support the health and care organisation to identify the scope (see task 2).

Task 2: Scoping the assessment

Conduct detailed scoping meeting to agree terms of reference

As part of the scoping meeting, the independent assessors and organisation should agree on the outcomes to be assessed during the assessment. There are 3 types of outcomes to be scoped:

There are 8 mandated outcomes to be audited for 2024/2025 CAF-aligned DSPT, which are listed below.
A2.a Risk management process  
A4.a Supply chain
B2.a Identity verification, authentication and authorisation
B4.d Vulnerability management
C1.a Monitoring coverage
D1.a Response plan
E2.b Consent
E3.a Using and sharing information sharing for direct care

Organisations are required to select a further 4 outcomes to be audited. Organisations and independent assessors need to discuss the risks faced by the organisation to decide on the most appropriate outcomes to be tested. These outcomes should be approved by the Board and will reflect areas of concern that warrant additional assurance over the controls in place during that audit period.   

The organisation may also choose to assess additional outcomes as part of this assessment. Additional outcome assessments, as well as the changes in budget and timelines, will need to be agreed with the independent assessors.

Understand any issues, incidents or changes since last DSPT independent assessment, and review the health and social care organisation’s action plan

We recommend that the independent assessors and the health and social care organisation review and discuss the previous year’s DSPT submission, as well as any recent issues, incidents and changes to inform the selection of the additional outcomes to be assessed. This should include reviewing the health and social care organisation’s action plan. 

Address any questions the organisation may have and ensure follow-up actions are included in the scope of the independent assessment

It is also recommended that any questions from the health and social care organisation are addressed during this meeting, or included in the scope of the assessment for resolution at a later date to ensure that all parties have the information they require ahead of the start of the assessment.

Agree with the organisation the level of access required, timings and availability of staff to assist in the independent assessment

The organisation and independent assessors will need to agree the timelines of the assessment, including the start date, expected end date for fieldwork and the submission date for the final report.

To ensure a prompt start to the assessment, we suggest that the independent assessors and the organisation discuss the documentation that will be required to verify the completeness of each outcome and underlying indicator of good practice. The assessed organisation should then aim to gather the documents ahead of the kick-off of the assessment. 

Agree terms of reference and receive formal sign off from sponsor of the review

Once the scope of the assessment is agreed, terms of reference should be documented (an example template is provided in Appendix B), finalised and receive formal sign off from the sponsor of the review.

Task 3: Performing the assessment

Kick-off assessment 

The kick-off plays a critical part in the CAF-aligned DSPT independent assessment as it is where the independent assessors are introduced to the organisation’s team. During this meeting, stakeholders will agree on the meetings to be held, review the list of documents to be provided, agree on the manner in which the documents will be provided to the independent assessors as well as the timelines to conduct those activities.

Review documents

Each document provided by the assessed organisation should be thoroughly reviewed by the independent assessors, with clear notes indicating the outcome and indicator of good practice that they relate to. This process may be iterative as the review of the original list may uncover the need for additional documentation to be obtained, which should be discussed during the interviews.

Performing interviews

The interview process allows the organisations to contextualise their processes and controls, giving independent assessors insights into the organisational structure, culture and the information technology (IT) architecture. It also gives independent assessors an insight into the organisation’s decision process and allows the independent assessors to ask additional questions based on the work already completed.

These interviews can also be used to assess the level of risk presented by a finding by discussing it with knowledgeable stakeholders.

Sample testing

As part of the independent assessment, many outcomes and indicators of good practice require the design of the controls to be tested.

However, testing every instance of a control is often not a practical or efficient way of ensuring that the control is adequately implemented. Instead, independent assessors will use sample testing to examine a representative portion of the population of data to draw conclusions about the entire population.

The number of samples to be tested will vary depending on several factors, such as the size of the overall population and the inherent risk of the control. Independent assessors will be required to use their professional judgement when choosing an adequate sample size. 

Testing techniques

The following testing techniques may be used, often in combination, to evaluate the design and/or operating effectiveness of a control:

1. Inquiry consists of seeking information, both financial and nonfinancial, of knowledgeable persons, within or outside the organisation. Inquiries may range from formal written inquiries to informal conversations. This testing technique provides the least assurance.
2. Observation consists of looking at a process or procedure being performed by others and observing how the control operates. This technique provides a higher level of assurance than inquiry alone.
3. Inspection involves examining records or documents, whether internal or external, in paper form, electronic form, or other media, or a physical examination of an asset, which support the operation of a control. Observations and inquiry will often be the 2 testing techniques used during stakeholder interviews. While they offer some assurance, inspecting documents that support the statements made during interviews offers additional assurance and is generally recommended, provided the documentation is available.
4. Automated testing or system query tests the operation of automated controls within an IT application, such as logical access configuration. This test can provide the most assurance if the evidence provided meets the requirements of the test. 

Documenting observations and issues

All notes made by the independent assessors during document reviews, interviews and sample tests should be carefully documented in a workbook, which clearly itemises the outcomes and indicators of good practice assessed and the result of the assessment, including observations and issues to be shared with the health and social care organisation.

Perform risk and confidence evaluations

Once all documents have been reviewed and all interviews have been held, the independent assessors should now have a list of findings, and should be able to assess each outcome against the 'Not achieved', 'Partially achieved' and 'Achieved'. Once each outcome has been evaluated, the independent assessor should then compare the results against the organisation’s self-assessment and obtain a confidence evaluation based on how closely the 2 assessments match.

Risk evaluation

Once the independent assessment provider has calculated the level of achievement for each outcome tested, the following principle can be used to allocate an overall risk assurance rating.

The independent assessment provider should calculate the overall risk rating of the organisation’s data security and protection control environment, for the in-scope assessments. The table below allows the independent assessment provider to conduct this calculation.

Confidence evaluation

Once the independent assessment provider has completed the fieldwork and concluded the level of achievement for each of the 12 outcomes tested and the overall risk evaluation, they can then determine the confidence-level in the veracity of the organisation’s CAF aligned DSPT self-assessment.

The self-assessment can be provided directly to the independent assessor or by viewing the latest DSPT submission at the time of the review. The independent assessor finding should be compared to the self-assessment to provide a confidence level in the self-assessment.

The following definitions should be used for aiding the decision of applying a confidence-level. It is noted that the evidence available to the independent assessor at the time of the assessment may differ or may have changed from the evidence in place at the time of the self-assessment.

Ensuring completeness of testing

Throughout the assessment, regular meetings are advised between the independent assessors and the health and social care organisation. This is to discuss their responses to indicators of good practice, establish the effectiveness of controls and processes in place, and ensure that outstanding requests for documentation, interviews and information are addressed promptly.

Task 4: Post assessment review meeting and reporting

Finalise observations hold close out meeting

The close out meeting is where the independent assessor should discuss findings, outcome ratings and confidence evaluations with the organisation to ensure both parties are aligned prior to the draft report being drafted. This gives an opportunity for the organisation to clarify any findings and to discuss where ratings differ from the self-assessment.

Draft and finalise report

Preparing a draft report

Reporting is a crucial part of the CAF-aligned DSPT independent assessment provider process and involves both verbal and written communication. Underpinning all of the CAF-aligned DSPT reporting and broader communications are the following principles:

‘No surprises’ – The CAF-aligned DSPT independent assessment provider will always ensure that findings are discussed with management prior to issuing draft reports. The CAF-aligned DSPT independent assessment provider will always seek to obtain full ‘buy in’ of management to recommendations to support successful implementation.

Clarity and consistency – The CAF-aligned DSPT independent assessment provider will avoid unnecessary jargon and will not shy away from setting out the key issues or themes arising from the work in clear, unambiguous terms;

Objectivity – The CAF-aligned DSPT independent assessment provider will use a standard scoring mechanism for all findings and for determining the overall rating of a report. This objective approach will be transparent and consistent across all reports.

Pragmatic and informed actions – The CAF-aligned DSPT independent assessment provider will not provide recommendations that run the risk of not being implemented. Rather, in the closing meeting of the audit, the CAF-aligned DSPT independent assessment provider will agree pragmatic, proportionate and realistic actions with the sponsor and include those in the CAF-aligned DSPT independent assessment provider report as the responses to each finding that is identified, along with responsible people and target dates for those actions.

Prioritisation – The format of reporting needs to provide a clear indication as to the relative importance of the issues being reported.

Coaching towards improvement – Discussion of the emerging findings, draft report and draft recommendations will afford the opportunity for the independent assessment provider to coach the organisation regarding good practice observed elsewhere and potential options for addressing controls weaknesses, and generally helping improve security and governance of information. This is a critical feature of the assessment as they should move the organisation towards achievement of improved security and governance of information outcomes; and the objective of safeguarding the confidentiality, integrity and availability of data assets.

The basic process for reporting after each assessment is shown below:

Issue tracking and follow-up work

All agreed recommendations arising from the work of the CAF-aligned DSPT independent assessment provider should be tracked to ensure their successful implementation. This is a critical element of the CAF-aligned DSPT independent assessment provider’s work and one which is not afforded the attention required in some organisations.

There are a number of ways that the CAF-aligned DSPT independent assessment provider can work with the health and social care organisation to ensure a slick and effective follow-up process. Typically, this might involve continued work with the health and social care organisation sponsor/director and/or the Head of Risk, regulation and performance to ensure the implementation of agreed actions resulting from CAF-aligned DSPT independent assessment provider reviews.

Typically, on an annual, bi-annual, or even quarterly basis, the CAF-aligned DSPT independent assessment provider should follow up on all due actions to verify management’s self-assessment of progress against these. This will involve looking at documentary evidence and re-performing testing. Recommendations will only be closed once the independent assessor is content that the action has been addressed in full and the risk mitigated.

In some cases, where there have been areas of specific concern raised or an identified need to re-assess the robustness of processes and controls the CAF-aligned DSPT, the independent assessment provider should also conduct specific follow-up reviews.

Task 5: Assessment finalisation and quality management

Follow-up discussions and tracking recommendations 

Co-ordinate follow-up discussions with the health and social care organisation to review progress on the agreed actions, and discuss queries that staff may have. As each action has a target deadline for implementation and completion, multiple discussions may be required to effectively track and monitor the organisation’s progress.

Skills and training

It is expected that during the reporting phase that knowledge gaps, learning and development needs are likely to be identified for the assessed organisation.

Independent organisations and assessment provider skills development

CAF-aligned DSPT independent assessment providers, whatever their status or background, will have personnel with training and development needs.

CAF-aligned DSPT independent assessment providers with new joiners or existing personnel who have never completed a NHS England DSPT independent assessment will need induction training to help them understand their role and the auditee organisation(s). All induction training is the responsibility of the employing organisation; be they a CAF-aligned DSPT independent organisation or an assessment service provider.

In particular, independent organisations and assessment providers with no prior experience of government, health and social care CAF-aligned DSPT independent assessments will need training to help them understand the health and social care sector accountability framework, especially those elements relating to governance and accountability.

It is recognised such organisations may have security and governance of information assessment, improvement capabilities and insights to share from other industries but it is imperative that they understand the health and social care sector. The task to understand the organisational profile and their operating environment is considered critical but even before this task is complete there is a baseline of sector knowledge that is needed before the security and governance of information knowledge of the independent assessor can be exploited to add value for the organisations assessed and the wider sector.

The independent organisation and assessment provider should ensure continuous learning plans are in place to develop existing personnel skills and ensure the organisation and provider stay current with the changing technology and threat landscape. In addition, the independent organisation and assessment provider should assist in the implementation of appropriate performance measurement systems.

Training needs analysis for independent assessors

Following release of NHS England’s mandated list of CAF-aligned DSPT outcomes to be audited in 2024/25, it is the responsibility of the independent organisation and assessment provider to consider the blend of skills, experience and seniority required to fulfil assessment against each outcome.

This can be achieved by conducting a training needs analysis (TNA) of independent assessment provider personnel. We encourage aligning this with the standard requirements of NCSC’s Cyber Resilience Audit (CRA) scheme. The scheme stipulates that the CRA service is overseen by a head consultant who holds a professional registration for the Cyber Audit and Assurance specialism Chartership title and the team leader delivering the audit holds a professional registration for the Cyber Audit and Assurance specialism Principle title (as a minimum), both titles awarded by the UK Cyber Security Council.

A TNA can help independent organisations or assessment providers understand whether there is sufficient capability and knowledge across their existing personnel to closely align to NHS England’s requirement for skills and competencies to deliver CAF-aligned DSPT independent assessment.

The TNA therefore helps the organisation or provider define the gap between the existing and the required skills and knowledge. The output articulates the:

• gap between current and required skills and knowledge
• general content of the required training, including learning methods and delivery of training