TINET: Transferring Knowledge between Invariant Networks

/in /by

Publication Date: 8/23/2018

Event: KDD 2018 – 24th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

Reference: pp. 1890-1899, 2018

Authors: Chen Luo, Rice University; Zhengzhang Chen, NEC Laboratories America, Inc.; Lu-An Tang, NEC Laboratories America, Inc.; Anshumali Shrivastava, Rice University; Jieping Ying, University of Michigan; Zhichun Li, NEC Laboratories America, Inc.; Haifeng Chen, NEC Laboratories America, Inc.

Abstract: The latent behavior of an information system that can exhibit extreme events, such as system faults or cyber-attacks, is complex. Recently, the invariant network has shown to be a powerful way of characterizing complex system behaviors. Structures and evolutions of the invariance network, in particular, the vanishing correlations, can shed light on identifying causal anomalies and performing system diagnosis. However, due to the dynamic and complex nature of real-world information systems, learning a reliable invariant network in a new environment often requires continuous collecting and analyzing the system surveillance data for several weeks or even months. Although the invariant networks learned from old environments have some common entities and entity relationships, these networks cannot be directly borrowed for the new environment due to the domain variety problem. To avoid the prohibitive time and resource consuming network building process, we propose TINET, a knowledge transfer based model for accelerating invariant network construction. In particular, we first propose an entity estimation model to estimate the probability of each source domain entity that can be included in the final invariant network of the target domain. Then, we propose a dependency construction model for constructing the unbiased dependency relationships by solving a two-constraint optimization problem. Extensive experiments on both synthetic and real-world datasets demonstrate the effectiveness and efficiency of TINET. We also apply TINET to a real enterprise security system for intrusion detection. TINET achieves superior detection performance at least 20 days lead-lag time in advance with more than 75% accuracy.

Publication Link: https://dl.acm.org/doi/10.1145/3219819.3220003