Curiosity-Guided Search refers to a concept in artificial intelligence and machine learning where an agent, typically in the context of reinforcement learning, actively explores its environment in a way that maximizes the acquisition of novel or informative experiences. The term “curiosity” in this context is used to describe the agent’s intrinsic motivation to seek out new and interesting information rather than relying solely on external rewards.

Curiosity-guided search has been applied in various domains, including robotics, video game playing, and autonomous systems. It is particularly useful in scenarios where the agent may encounter sparse or delayed external rewards, as the intrinsic motivation to explore allows it to gather knowledge and build a more nuanced understanding of the environment.

The implementation of curiosity-guided search often involves designing algorithms that quantify and encourage exploration, taking into account uncertainty, novelty, or other metrics associated with information gain. This approach aligns with the idea that agents with a curiosity-driven mindset can learn more efficiently and adapt to a wider range of situations.

Posts

Automated Anomaly Detection via Curiosity-Guided Search and Self-Imitation Learning

/in /by

Anomaly detection is an important data mining task with numerous applications, such as intrusion detection, credit card fraud detection, and video surveillance. However, given a specific complicated task with complicated data, the process of building an effective deep learning-based system for anomaly detection still highly relies on human expertise and laboring trials. Also, while neural architecture search (NAS) has shown its promise in discovering effective deep architectures in various domains, such as image classification, object detection, and semantic segmentation, contemporary NAS methods are not suitable for anomaly detection due to the lack of intrinsic search space, unstable search process, and low sample efficiency. To bridge the gap, in this article, we propose AutoAD, an automated anomaly detection framework, which aims to search for an optimal neural network model within a predefined search space. Specifically, we first design a curiosity-guided search strategy to overcome the curse of local optimality. A controller, which acts as a search agent, is encouraged to take actions to maximize the information gain about the controller’s internal belief. We further introduce an experience replay mechanism based on self-imitation learning to improve the sample efficiency. Experimental results on various real-world benchmark datasets demonstrate that the deep model identified by AutoAD achieves the best performance, comparing with existing handcrafted models and traditional search methods.

Read more

AutoOD: Neural Architecture Search for Outlier Detection

/in /by

Outlier detection is an important data mining task with numerous applications such as intrusion detection, credit card fraud detection, and video surveillance. However, given a specific task with complex data, the process of building an effective deep learning based system for outlier detection still highly relies on human expertise and laboring trials. Moreover, while Neural Architecture Search (NAS) has shown its promise in discovering effective deep architectures in various domains, such as image classification, object detection and semantic segmentation, contemporary NAS methods are not suitable for outlier detection due to the lack of intrinsic search space and low sample efficiency. To bridge the gap, in this paper, we propose AutoOD, an automated outlier detection framework, which aims to search for an optimal neural network model within a predefined search space. Specifically, we introduce an experience replay mechanism based on self-imitation learning to improve the sample efficiency. Experimental results on various real-world benchmark datasets demonstrate that the deep model identified by AutoOD achieves the best performance, comparing with existing handcrafted models and traditional search methods.

Read more