GLAD: Content-Aware Dynamic Graphs for Log Anomaly Detection

/in /by

Publication Date: 12/2/2023

Event: IEEE International Conference On Knowledge Graph (ICKG-2023), Shanghai, China

Reference: pp. 1-10, 2023

Authors: Yufei Li, University of California, Riverside; Yanchi Liu, NEC Laboratories America, Inc.; Haoyu Wang, NEC Laboratories America, Inc.; Zhengzhang Chen, NEC Laboratories America, Inc.; Wei Cheng, NEC Laboratories America, Inc.; Yuncong Chen, NEC Laboratories America, Inc.; Wenchao Yu, NEC Laboratories America, Inc.; Haifeng Chen, NEC Laboratories America, Inc.; Cong Liu, University of California, Riverside

Abstract: Logs play a crucial role in system monitoring and debugging by recording valuable system information, including events and status. Although various methods have been proposed to detect anomalies in log sequences, they often overlook the significance of considering relationships among system components, such as services and users, which can be identified from log contents. Understanding these relationships is vital for identifying anomalies and their underlying causes. To address this issue, we introduce GLAD, a Graph-based Log Anomaly Detection framework designed to detect relational anomalies in system logs. GLAD incorporates log semantics, relationship patterns, and sequential patterns into a unified framework for anomaly detection. Specifically, GLAD first introduces a field extraction module that utilizes prompt-based few-shot learning to extract essential field information, such as services and users, from log contents. Then GLAD constructs dynamic log graphs for sliding windows by leveraging the log events and extracted fields. These graphs represent events and fields as nodes and their relationships as edges. Subsequently, we propose atemporal-attentive graph edge anomaly detection model for identifying anomalous relationships in the dynamic log graphs. This model employs a Graph Neural Network (GNN)-based encoder enhanced with transformers to capture structural, content, and temporal features. We evaluate our proposed method on three datasets, and the results demonstrate the effectiveness of GLAD in detecting anomalies indicated by varying relation patterns.

Publication Link: https://ieeexplore.ieee.org/document/10412723

Additional Publication Link: https://arxiv.org/pdf/2309.05953.pdf