Frank Duff Chats about Latest MITRE ATT&CK Evaluation

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | RSS

TechSpective Podcast Episode 056

MITRE ATT&CK has quickly become an invaluable, third-party assessment that the cybersecurity industry and customers can reference as an objective demonstration of how the tools and platforms perform in real-world conditions. MITRE just released the results of the third MITRE ATT&CK Evaluation. 29 vendors participated in the exhaustive exercise to validate the capabilities of their products against Carbanak and Fin7–two threat groups that are known for using innovating techniques and tactics to target the financial sector.

Frank Duff, Director of ATT&CK Evaluations at MITRE Engenuity / Group Lead at MITRE, joined me for this episode of the TechSpective Podcast to talk about the origins, objectives, and value of the MITRE ATT&CK Evaluations.

One of the unique interesting things about the MITRE ATT&CK Evaluations is that MITRE doesn’t declare a “winner” per se. It turns out that is actually both a pro and a con. On the one hand, it is admirable that MITRE chooses to approach the evaluation clinically and simply share the results and data to let customers review it and come to their own conclusions. The downside is that no clear or declared “winner” provides virtually every participating vendor an opportunity to spin the results so they somehow appear to have crushed the competition. It is easy to get “creative” about what you focus on or show, or conveniently exclude a vendor that was very competitive overall and even beat you in some areas from your talking points.

The marketing hype from the vendors can be confusing, but at least customers can still fall back on the actual MITRE ATT&CK Evaluation data and draw their own conclusions. There is also help from respected independent sources like former Forrester Analyst Josh Zelonis. Each year he has shed light on the MITRE ATT&CK Evaluation results and he provides source code for a tool to enable companies to generate their own custom analysis of the results.

Listen to the podcast for more insight on how the MITRE ATT&CK Evaluations are conducted, and how they evolve year over year based on previous tests and current attack trends. Please ask questions and share your thoughts on the topic in the comments below. I appreciate your help to share the podcast and grow the audience. Also, please subscribe to the TechSpective Podcast through your favorite podcast platform, and share the podcast with your peers and friends.

If you enjoy the podcast, I would also be grateful if you could take 2 minutes to rate and review the podcast on iTunes, or wherever you listen.