LLM-Based Entity Extraction for Cyber Threat Reports
Name
LLMBasedEntityExtractionforCTRs_deSilva2025.pdf
Description
Technical Report
Size
509.07 KB
Format
Adobe PDF
Checksum (MD5)
11863bf48cc6d8c720f4c52ee56333d0
Author(s)
Alperin, Kenneth
de Silva, Alexis
Date Issued
September 10, 2025
Abstract
As the cyber threat landscape and capabilities
of advance persistent threats continue to expand, applying
cutting-edge technology to the domain of cyber intelligence
is necessary for the United States Space Force
to keep pace in the Great Power Competition. Cyber
intelligence analysts spend an estimated time of nearly
840 man-hours annually on the extraction and validation
of relevant intelligence from cyber threat reports (CTRs).
Named entity recognition (NER) is a natural language
processing technique capable of automatically extracting
and labeling all relevant information from a given text.
Although not a novel idea, this paper aims to expand
the current but limited research on the applications of
NER to the domain of cyber intelligence. This study
uses a new openly-licensed dataset, AnnoCTR, to finetune
a cybersecurity-specific, transformers-based model,
CYBERT. The performance of the model is compared
to the models from the derived literature. Although the
results showed an F1 score of 0.733 – a less optimal
performance compared to previous models – there is
still more work to explore to reduce the production time
of intelligence analysis by half.
Subjects
Artificial Intelligence
LLSC
Machine Learning
Intelligence community
MIT Department
Lincoln Laboratory
Persistent DSpace Link
