Principle D2: Lessons learned

When an incident or near miss occurs, steps are taken to understand its causes and to ensure remediating action is taken to protect against future incidents.

Description

When an incident or near miss occurs, your organisation takes steps to understand its causes, informing appropriate remediating action.

Indicators of good practice (IGP) achievement levels can be viewed via the Data Security Protection Toolkit.

As documented in the introduction to this framework, independent assessors are expected to use their professional judgement when assessing organisations against the Cyber Assessment Framework (CAF). 

The approach and documentation list described below are a suggestion to the independent assessors, and do not have to be followed stringently.

Suggested approach to testing

1. Routinely performing post incident analysis – Assess whether post incident analysis is documented as a key step in the response process for both incidents and near misses. Verify that the organisation follows criteria for determining when a lessons learned exercise is required, ensuring that reviews are proportionate to the nature, impact and learning potential of the incident or near miss. (A#1)

2. Comprehensive post incident analysis - Verify that the scope of the organisation’s post-incident analysis considers organisational factors, technical factors, human factors and threat factors where relevant to the incident or near miss being investigated. (A#2)

3. Making incident or near miss data available – Obtain evidence that relevant incident or near miss data, such as incident records, system logs, monitoring information, audit trails, communications records, threat intelligence, technical evidence and supporting documentation, are routinely made available to those undertaking the post-incident review. (A#3)

4. ‘What if’ / ‘if only’ scenarios for cyber security incidents – For cyber security incidents only, assess whether reviewers are required to consider plausible alternative circumstances that could have increased the impact of an incident or near miss. Verify that the post-incident analysis includes prompts, discussion activities or other mechanisms to support "what if" or "if only" analysis. (A#4)

Suggested documentation

Suggested documentation should include:

• criteria or methodology for determining when post-incident analysis is required
• completed post-incident reviews from the past 12 months demonstrating consideration of organisational, human, technical and threat factors
• evidence that relevant incident and near miss data is routinely made available to post-incident reviewers
• evidence of consideration of 'what if' / 'if only' prompts

Description

Your organisation uses lessons learned from incidents and near misses to improve your security measures.

Indicators of good practice (IGP) achievement levels can be viewed via the Data Security Protection Toolkit.

As documented in the introduction to this framework, independent assessors are expected to use their professional judgement when assessing organisations against the Cyber Assessment Framework (CAF). 

The approach and documentation list described below are a suggestion to the independent assessors, and do not have to be followed stringently.

Suggested approach to testing

1. Incident review process or policy - Obtain the incident review process or policy, and assess whether the document contains:

1. A requirement for lessons learned to be undertaken for near misses as well as incidents. (A#1)
2. Assigned responsibilities for capturing lessons learned, updating relevant processes and documentation, and disseminating the learning throughout the organisation. (A#1)
3. A documented authority to review the outputs of lessons learned exercises and direct the improvements in security measures. (A#3)
4. A clearly defined prioritisation process for the identified security improvements, including ownership for implementing changes and the process for approval of changes. (A#4)
5. The escalation process and reporting lines to senior management. (A#5)  

2. Lessons learned activities - Assess whether the scope of the lessons learned activities include a review of reporting, roles, governance, skills and organisational processes as well as technical aspects of networks and information systems (A#2).

3. Incident sampling - Obtain the list of incidents that have taken place in the last 12 months. From that list, choose a sample (see Guide for CAF-aligned DSPT independent assessors for more information on sampling) and request to see the respective lessons learned exercise. Verify that:

1. The scope of the samples matches the expected scope as per the incident review process/policy, as assessed during step 2. (A#2)
2. The findings of the samples have been documented, with remediation actions designed and their implementation assigned to a named owner with adequate timelines. (A#3)

4. Re-testing - Enquire of any plans to re-test the response plans with the updated security measures, where necessary and obtain evidence that this test is being designed. (A#3)

5. Mitigating actions - Obtain and inspect documentation showing the progress that has been made on the implementation of mitigation actions. This may include updates to policies and process documentation, but also technical changes to security systems as required. (A#4)

6. Approval of mitigating actions - Obtain and inspect the terms of reference and minutes of the responsible group(s) to verify that outputs of the lessons learned exercises are being discussed, reviewed and approved by the responsible group(s). The remediation actions should be prioritised and approved by a relevant authority, with a named owner and adequate timelines put in place. (A#1)

7. Incorporating into risk management and continuous improvement - Obtain examples of where lessons learned exercise and root cause analysis from incidents and near misses have been incorporated into risk management and continuous improvement. (A#5)

8. Using 'what if' / 'if only' scenarios to drive improvements – For cyber incidents only, assess whether plausible alternative scenarios identified through 'what if' or 'if only' analysis have been used to identify improvements to the organisation’s security, resilience and response arrangements. Assess whether the outputs of these discussions have been translated into specific improvement actions, assigned to named owners and delivered. (A#6)

9. Learning from reported incidents – Review a sample of external incident reports, alerts or advisories received by the organisation. Assess whether the organisation has documented its consideration of the information, identified any required improvements to its own technologies and services, assigned actions to appropriate owners and incorporated relevant learning into its own security controls, processes and policies. (A#7)

Suggested documentation

Suggested documentation includes:

• incident review process/policy
• documented lessons learned and post incident analysis activities
• evidence of methodology and considerations for undertaking post incident analysis
• list of incidents from the last 12 months
• evidence of actions to take following lessons learned activities
• evidence of planning to re-test response plans or evidence that re-testing has occurred
• evidence of policies, processes and systems being updated following lessons learned activities
• terms of reference and minutes of relevant groups
• evidence of risk management processes being updated following lessons learned activities
• improvement plans, action logs or remediation trackers arising from 'what if' / 'if only' analysis
• evidence of updates to security controls, policies, processes, testing programmes, incident response arrangements or resilience plans following consideration of external incidents