Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection

/in /by

Publication Date: 5/3/2018

Event: Proceedings of the 6th International Conference on Learning Representations, Vancouver Convention Center (ICLR 2018)

Reference: pp. 1-19, 2018

Authors: Bo Zong, NEC Laboratories America, Inc.; Qi Song, NEC Laboratories America Inc.; Washington State University; Martin Renqiang Min, NEC Laboratories America, Inc.; Wei Cheng, NEC Laboratories America, Inc.; Cristian Lumezanu, NEC Laboratories America, Inc.; Daeki Cho, NEC Laboratories America, Inc.; Haifeng Chen, NEC Laboratories America, Inc.

Abstract: Unsupervised anomaly detection on multi- or high-dimensional data is of great importance in both fundamental machine learning research and industrial applications, for which density estimation lies at the core. Although previous approaches based on dimensionality reduction followed by density estimation have made fruitful progress, they mainly suffer from decoupled model learning with inconsistent optimization goals and incapability of preserving essential information in the low-dimensional space. In this paper, we present a Deep Autoencoding Gaussian Mixture Model (DAGMM) for unsupervised anomaly detection. Our model utilizes a deep autoencoder to generate a low-dimensional representation and reconstruction error for each input data point, which is further fed into a Gaussian Mixture Model (GMM). Instead of using decoupled two-stage training and the standard Expectation-Maximization (EM) algorithm, DAGMM jointly optimizes the parameters of the deep autoencoder and the mixture model simultaneously in an end-to-end fashion, leveraging a separate estimation network to facilitate the parameter learning of the mixture model. The joint optimization, which well balances autoencoding reconstruction, density estimation of latent representation, and regularization, helps the autoencoder escape from less attractive local optima and further reduce reconstruction errors, avoiding the need of pre-training. Experimental results on several public benchmark datasets show that, DAGMM significantly outperforms state-of-the-art anomaly detection techniques, and achieves up to 14% improvement based on the standard F1 score.

Publication Link: https://openreview.net/pdf?id=BJJLHbb0-