Enhancing a Deep Learning Model for Anomaly Detection in Time-Series Data Using Attention Networks
Abstract
Anomaly detection in time-series data is essential for monitoring industrial machinery and computing systems, yet labelled anomalies are scarce and real signals are often periodic and non-stationary. This paper presents the Attention-based Temporal Reconstruction Network (ATRN), an unsupervised Transformer encoder–decoder with a low-dimensional window bottleneck that prevents identity mapping, and an overlap-averaging scheme that converts window reconstruction errors into point-level anomaly scores. ATRN was evaluated on seven real-world series from the Numenta Anomaly Benchmark (realKnownCause) against Isolation Forest, an LSTM autoencoder and an attention-free ablation, under two label-free training protocols and three random seeds, using ROC-AUC, PR-AUC and best F1 without point adjustment. When trained on a clean initial segment, ATRN achieved the highest mean ROC-AUC among the neural models (0.695 vs. 0.660 for the LSTM autoencoder), and attention improved ROC-AUC on the most periodic series by 0.091. However, Isolation Forest obtained the best overall accuracy (mean ROC-AUC 0.750), training-set contamination removed the advantage of attention, and the learned attention maps were largely diffuse. The results indicate that attention is most useful when anomalies disrupt regular temporal patterns, and they underline the importance of strong baselines.