Achieving a holistic view of security operations performance is essential for organizations to make informed decisions and demonstrate the impact on business outcomes.
Most cybersecurity metrics available come from individual security tools. The data remains isolated within respective silos, restricting the ability to gain a holistic understanding of your security program. The number of patches applied, number of alerts per day, and host infections to date carry significance, but not for the board. They fail to convey the necessary insights to understand the value their security operations bring to the organization.
In June 2023, Gartner stated, “There are many ways to capture and communicate the value of cybersecurity in business terms.”
To truly assess the effectiveness of cybersecurity efforts, organizations need to go beyond individual metrics and embrace a holistic approach. One way to do so is by aligning it with your security operations workflow, defined as the detection, investigation, and response (DIR) process or lifecycle.
Holistic Measurement of DIR Workflow
With a focus on the DIR lifecycle, significant advancements can be made in understanding how to enhance visibility in their environment, expand threat coverage, and improve team performance. Measuring the areas below helps evaluate the effectiveness of the overall DIR process and informs decisions around your team’s performance.
1. Detection: Visibility into Data Sources
Effective security operations require comprehensive detection visibility across various data sources, including SIEMs, clouds, and EDRs. Restricting visibility to just some of these sources can create blind spots, leaving room for adversaries to exploit vulnerabilities. It is crucial to understand what activities you need to monitor (log diversity) and where these activities are being monitored (log coverage).
You should ask:
- Do you have a diverse range of log sources that are capable of detecting attacker activity across the environment at different stages of the kill chain? For example, do you have a secure email gateway to block known malicious emails, EDR technology to identify any malware being executed from phishing emails, and a proxy to monitor for any exfiltrated data?
- Do you have coverage of a specific log source for your entire business infrastructure? For example, if you have firewalls, do they cover all segments of your corporate network? Are all of them being centralized into a central location like a SIEM?
2. Detection: Coverage Mapped to Risk Frameworks
The measurement of detection coverage involves assessing the level of visibility within your environment, including the types of attacks accurately detected and those that may go unnoticed. An effective way to convey this information is to align your detection efforts with the tactics and techniques outlined in MITRE ATT&CK.
Once you’ve identified the gap between known threats and your existing detections, the next step is to determine which of these gaps are relevant to your business. By prioritizing the detection of techniques that are applicable to your specific environment, you can allocate your resources effectively and strengthen your defenses.
For instance, if your business operates solely on-premises and utilizes SaaS apps without any cloud infrastructure, detecting techniques targeting the cloud may not be a top priority. For high-priority gaps, you can develop future roadmaps that include new detections and address any associated log source gaps.
You should ask:
- Does your coverage align with the tactics and techniques outlined in MITRE, specifically targeting those that most disrupt your business?
- How do you compare against your peers and industry?
- Do you have detections in place around your crown jewels/critical infrastructure?
- What are your false positive and anomalous safe alert rates?
3. Investigation: Proactive Capabilities
Measuring your team’s investigation process allows you to gain insights into your ability to proactively identify and investigate suspicious behavior in your environment. Here, you can identify shortfalls in investigative capabilities, address alert noise, and improve proactivity in threat identification and mitigation.
Given the importance of speed and accuracy in this context, the most significant enhancements to metrics like Mean Time to Resolve (MTTR) are achieved by minimizing unnecessary distractions.
There are several methods to achieve this, including automating repetitive tasks, establishing clear communication channels and responsibilities through streamlined processes, and ensuring that alerts triggered in production maintain a high level of accuracy.
First, false positives can distract your team from focusing on substantive investigations, resulting in wasted time and resources.
Secondly, anomalous safe alerts, often confused with false positives, should be tracked to ensure your detection rules detect potential risky events among those that are expected and marked “safe.”
Thirdly, understanding your threat-hunting frequency and effectiveness gives you a sense of your team’s proactive capabilities. This can include regularly hunting for threats, leveraging threat intelligence, and actively seeking out potential weaknesses in your environment.
You should ask:
- How long does it take you to complete an investigation from the time of an alert?
- How many automated investigation and response actions do you take?
- What are your false positive and anomalous safe alert rates?
- What’s your frequency of successful threat hunting?
4. Response: Speed of the DIR Process
Response can be seen as the ability of a SOC or incident response team to efficiently validate and address a security incident based on detection. You want the ability to analyze the overall process and locate areas for improvement. Response metrics allow you to better understand if the DIR process has been optimally streamlined, whether you are properly staffed, if your team needs training, and if the problem is getting the business to act.
You should ask:
- How long does it take you to remediate an incident once it happens?
- What is your average dwell time?
- What’s your rate of deployed response playbooks?
- What is your ratio of open to closed alerts?
Reaching the Board & Prioritizing Change
By using holistic security measurements that align with your organization’s DIR workflow, you can effectively address questions from business leaders regarding security performance and the value of their investments.
As a security leader, utilizing these holistic measurements allows you to assess the overall security posture of the organization, identify potential risks, and prioritize projects accordingly.
