Multimodal Learning is a machine learning approach that combines and processes information from multiple types of data, such as text, images, audio, or video. By integrating different modalities, models can capture richer relationships and context than using a single data source. Multimodal learning is widely used in applications such as vision language models, speech recognition, and multimedia analysis.

Posts

Celebrating the Women of NEC Laboratories America

NEC Laboratories America celebrates Women’s History Month and International Women’s Day by recognizing the women researchers, engineers, and interns whose work in artificial intelligence, optical networking, cybersecurity, and data science is helping shape the future of technology.

Online Multi-modal Root Cause Identification in Microservice Systems

Root Cause Analysis (RCA) is essential for pinpointing the root causes of failures in microservice systems. Traditional data-driven RCA methods are typically limited to offline applications due to high computational demands, and existing online RCA methods handle only single-modal data, overlooking complex interactions in multi-modal systems. In this paper, we introduce OCEAN, a novel online multi-modal causal structure learning method for root cause localization. OCEAN introduces a long-term temporal causal learning module with two encoders: one captures stable causal dependencies from historical data, while the other models short-term variations in the current batch data. We further design a multi-factor attention mechanism to analyze and reassess the relationships among different metrics and log indicators/attributes for enhanced online causal graph learning. Additionally, a contrastive mutual information maximization-based graph fusion module is developed to effectively model the relationships across various modalities. Extensive experiments on three real-world datasets demonstrate the effectiveness and efficiency of our proposed method.