  Semi-supervised anomaly detection for sensor signals is critical in ensuring
system reliability in smart manufacturing. However, existing methods rely
heavily on data correlation, neglecting causality and leading to potential
misinterpretations due to confounding factors. Moreover, while current
reinforcement learning-based methods can effectively identify known and unknown
anomalies with limited labeled samples, these methods still face several
challenges, such as under-utilization of priori knowledge, lack of model
flexibility, and deficient reward feedback during environmental interactions.
To address the above problems, this paper innovatively constructs a
counterfactual causal reinforcement learning model, termed Triple-Assisted
Causal Reinforcement Learning Anomaly Detector (Tri-CRLAD). The model leverages
causal inference to extract the intrinsic causal feature in data, enhancing the
agent's utilization of prior knowledge and improving its generalization
capability. In addition, Tri-CRLAD features a triple decision support
mechanism, including a sampling strategy based on historical similarity, an
adaptive threshold smoothing adjustment strategy, and an adaptive decision
reward mechanism. These mechanisms further enhance the flexibility and
generalization ability of the model, enabling it to effectively respond to
various complex and dynamically changing environments. Experimental results
across seven diverse sensor signal datasets demonstrate that Tri-CRLAD
outperforms nine state-of-the-art baseline methods. Notably, Tri-CRLAD achieves
up to a 23\% improvement in anomaly detection stability with minimal known
anomaly samples, highlighting its potential in semi-supervised anomaly
detection scenarios. Our code is available at
https://github.com/Aoudsung/Tri-CRLAD.
