  Large Language Models (LLMs) have demonstrated remarkable capabilities in
various domains, but their vulnerability to trojan or backdoor attacks poses
significant security risks. This paper explores the challenges and insights
gained from the Trojan Detection Competition 2023 (TDC2023), which focused on
identifying and evaluating trojan attacks on LLMs. We investigate the
difficulty of distinguishing between intended and unintended triggers, as well
as the feasibility of reverse engineering trojans in real-world scenarios. Our
comparative analysis of various trojan detection methods reveals that achieving
high Recall scores is significantly more challenging than obtaining high
Reverse-Engineering Attack Success Rate (REASR) scores. The top-performing
methods in the competition achieved Recall scores around 0.16, comparable to a
simple baseline of randomly sampling sentences from a distribution similar to
the given training prefixes. This finding raises questions about the
detectability and recoverability of trojans inserted into the model, given only
the harmful targets. Despite the inability to fully solve the problem, the
competition has led to interesting observations about the viability of trojan
detection and improved techniques for optimizing LLM input prompts. The
phenomenon of unintended triggers and the difficulty in distinguishing them
from intended triggers highlights the need for further research into the
robustness and interpretability of LLMs. The TDC2023 has provided valuable
insights into the challenges and opportunities associated with trojan detection
in LLMs, laying the groundwork for future research in this area to ensure their
safety and reliability in real-world applications.
