Experience Replay is a technique used in reinforcement learning, specifically in the context of training artificial intelligence (AI) agents, such as deep neural networks, to improve learning efficiency and stability. It was introduced to address certain challenges associated with the training of deep reinforcement learning models. In the standard reinforcement learning setup, an agent interacts with an environment, takes actions, receives rewards, and learns from this interaction. Experience replay introduces the concept of storing and reusing past experiences (tuples of state, action, reward, next state) in a replay buffer.

Posts

Automated Anomaly Detection via Curiosity-Guided Search and Self-Imitation Learning

/in /by

Anomaly detection is an important data mining task with numerous applications, such as intrusion detection, credit card fraud detection, and video surveillance. However, given a specific complicated task with complicated data, the process of building an effective deep learning-based system for anomaly detection still highly relies on human expertise and laboring trials. Also, while neural architecture search (NAS) has shown its promise in discovering effective deep architectures in various domains, such as image classification, object detection, and semantic segmentation, contemporary NAS methods are not suitable for anomaly detection due to the lack of intrinsic search space, unstable search process, and low sample efficiency. To bridge the gap, in this article, we propose AutoAD, an automated anomaly detection framework, which aims to search for an optimal neural network model within a predefined search space. Specifically, we first design a curiosity-guided search strategy to overcome the curse of local optimality. A controller, which acts as a search agent, is encouraged to take actions to maximize the information gain about the controller’s internal belief. We further introduce an experience replay mechanism based on self-imitation learning to improve the sample efficiency. Experimental results on various real-world benchmark datasets demonstrate that the deep model identified by AutoAD achieves the best performance, comparing with existing handcrafted models and traditional search methods.

Read more