MBuechel/TTP_SOK

SoK: Automated TTP Extraction from CTI Reports – Are We There Yet?

This repository contains the pre-trained models of the paper "SoK: Automated TTP Extraction from CTI Reports – Are We There Yet?" [1] PDF.

Introduction

Cyber Threat Intelligence (CTI) plays a critical role in sharing knowledge about new and evolving threats. With the increased prevalence and sophistication of threat actors, intelligence has expanded from simple indicators of compromise to extensive CTI reports describing high-level attack steps known as Tactics, Techniques and Procedures (TTPs). Such TTPs, often classified into the ontology of the ATT&CK framework, make CTI significantly more valuable, but also harder to interpret and automatically process. Natural Language Processing (NLP) makes it possible to automate large parts of the knowledge extraction from CTI reports; over 40 papers discuss approaches, ranging from named entity recognition over embedder models to generative large language models. Unfortunately, existing solutions are largely incomparable as they consider decisively different and constrained settings, rely on custom TTP ontologies, and use a multitude of custom, inaccessible CTI datasets. We take stock, systematize the knowledge in the field, and empirically evaluate existing approaches in a unified setting for fair comparisons. We gain several fundamental insights, including (1) the finding of a kind of performance limit that existing approaches seemingly cannot overcome as of yet, (2) that traditional NLP approaches (possibly counterintuitively) outperform modern embedder-based and generative approaches in realistic settings, and (3) that further research on understanding inherent ambiguities in TTP ontologies and on the creation of qualitative datasets is key to take a leap in the field.

Organization

Our code and further instructions for it can be found in our Zenodo code repository at: https://doi.org/10.5281/zenodo.15608555

This repository contains “only” the pre-trained models used in the paper.

References

[1] Marvin Büchel, Tommaso Paladini, Stefano Longari, Michele Carminati, Stefano Zanero, Hodaya Binyamini, Gal Engelberg, Dan Klein, Giancarlo Guizzardi, Marco Caselli, Andrea Continella, Maarten van Steen, Andreas Peter, Thijs van Ede. (2025, August). SoK: Automated TTP Extraction from CTI Reports – Are We There Yet? In 34th USENIX Security Symposium (USENIX). USENIX.

Bibtex

@inproceedings{buechel2025sok,
  title={{SoK: Automated TTP Extraction from CTI Reports – Are We There Yet?}},
  author={Büchel, Marvin and Paladini, Tommaso and Longari, Stefano and Carminati, Michele and Zanero, Stefano and Binyamini, Hodaya and Engelberg, Gal and Klein, Dan and Guizzardi, Giancarlo and Caselli, Marco and Continella, Andrea and van Steen, Maarten and Peter, Andreas and van Ede, Thijs},
  booktitle={34th USENIX Security Symposium (USENIX)},
  year={2025},
  organization={USENIX}
}