Header

Hero Section

Machine Learning

Projects

Projects

Text Analysis

Deep learning architectures handle various natural language processing tasks, including parsing, part-of-speech tagging, chunking, named-entity recognition and semantic role labeling. Instead of depending on hand-crafted features that are engineered for specific tasks, the system we have developed learns internal representations from mostly unlabeled training data. This makes the system very flexible, since it can be trained for a different language simply by training with text data from that language. So far we have developed systems for English, Japanese and Chinese, achieving state-of-the-art or better performance in all cases.

READ MORE

Torch 7: Open Source System

This system provides a powerful environment for state-of-the-art machine learning algorithms. It is easy to use and provides a very efficient implementation, thanks to the fast scripting language (Lua) and underlying C implementations.

READ MORE

High-Order Feature Learning

High-order feature interactions can capture intuitively understandable structures in data of interest. The high-order parametric embedding (HOPE) is an efficient algorithm to determine high-order features, generating data embeddings suitable for visualization. Compared to deep embedding models with complicated architectures, HOPE is considerably more effective in learning high-order feature mappings, and it can also synthesize a small number of exemplars to represent the entire dataset in a low-dimensional way. To compute this efficiently, we have developed novel techniques based on tensor factorization. Our approach is applicable to a wide variety of problems where interpretation of the trained models is important.

READ MORE

eigen Video Understanding

Aiming to achieve near-human video understanding, in this project we analyze several gigabytes of spatiotemporal data to perform action recognition, multi-person tracking, object permanence and video reasoning. eigen has built a scalable video-understanding platform for long-form video reasoning that scales to new environments and camera angles without any re-training. eigen also provides a system platform. Running both on the cloud (AWS) and on-prem, it can scale up to thousands of streams into it for cloud-based AI processing. Our AI video algorithms provide efficient streaming and inference. eigen has a web frontend and support for iOS/Android playback. eigen has been tested in several retail POCs serving 200+ streams; its behavioral analytics have also been evaluated through various NEC customers. Using mixed precision and TensorRT, eigen is extremely efficient and incurs very low cloud costs. PublicationsHopper: Multi-hop Transformer for Spatiotemporal Reasoning. In ICLR 2021. PDFPoster 15 Keypoints is All You Need. Michael Snower, Asim Kadav, Farley Lai, Hans Peter Graf. In CVPR 2020. PDFRanked #1 in PoseTrack Tripping Through Time: Efficient Localization of Activities in Videos. (Spotlight) Meera Hahn, Asim Kadav, James M. Rehg, Hans Peter Graf. In CVPR Workship on Language and Vision, 2019. Also, appears in BMVC'20. PDF Visual Entailment: A Novel Task for Fine-Grained Image Understanding. Ning Xie, Farley Lai, Derek Doran, Asim Kadav. In NeurIPS Workshop on Visually-Grounded Interaction and Language, 2018 (ViGIL’18). PDFDatasetleaderboard Teaching Syntax using Adverserial Distraction. Juho Kim, Christopher Malon, and Asim Kadav. In FEVER-EMNLP, 2018. PDF Attend and Interact: Higher-Order Object Interactions for Video Understanding. Chih-Yao Ma, Asim Kadav, Iain Melvin, Zsolt Kira, Ghassan AlRegib, and Hans Peter Graf. In CVPR, 2018. PDFVideoLive Demo

READ MORE

Machine Learning Development Environment

Data analytics requires a wide range of tools for a wide range of tasks: data collection, cleaning and labeling; model training and testing; presenting results, etc. Our system is designed for both performance and ease of use. The software architecture is cleanly structured into layers, with the lowest one providing math and statistics functions, followed by a layer of machine learning and data handling tools. Templates for application domains are contained in the next layer, implemented in script language (Lua). These templates are adapted to customer requirements by field engineers when deploying applications at a customer premise.

READ MORE

Digital Pathology

Accurate and fast diagnosis based on histological samples is crucial for prevention, early detection and treatment of cancer. We have developed a digital pathology system that uses machine learning algorithms to analyze images of tissue to assist in cancer diagnosis. This system is used in major diagnostics laboratories in Japan for quality control.

READ MORE

Deep Learning

We have been developing neural network learning algorithms for more than a decade, and several of the most successful algorithms in use today have been created at NEC Labs. These include original algorithms for image/video interpretation and text analysis. Our focus is on flexible algorithms that can handle any type of data and deal with large-scale problems.

READ MORE

Machine Learning Parallelization

Large-scale data analytics is compute intensive and requires parallelization of algorithms as well as optimization of the data flow. We develop various types of parallelizations for multi-core systems and clusters. We also work with heterogeneous systems that include GPUs or vector processors. MALT is one of our projects that enables parallelization over a large number of processors through virtual shared memory. MALT provides abstractions for fine-grained in-memory updates using one-sided RDMA, limiting data movement costs during incremental model updates. Developers can specify the dataflow while MALT takes care of communication and representation optimizations. Machine learning applications, written in C, C++ and Lua, are supported based on SVM, matrix factorization and deep learning. Besides speedup, MALT also provides fault tolerance and guarantees network efficiency. We are implementing various new distributed optimization algorithms on MALT, such as RWDDA and support for multiple GPUs.

READ MORE

Semantic Analysis & Reasoning

We develop several types of algorithms for high-level semantic analysis, which are used for tasks such as scene interpretation, document retrieval and question answering. For text interpretation, a syntactic analysis extracts relevant elements followed by concept interpretation. To combine different data types, a data-specific module first generates metadata representations that are integrated into a deep learning network. Supervised sequence embedding (SSE) is a simple and efficient technique to interpret shorter segments of text, such as product reviews or e-mail messages. Short phrases (n-grams) are modeled in a latent space. The phrases are then combined to form document-level latent representations, where position of an n-gram in the document is used to compute corresponding combining weight. The resulting two-stage supervised embedding is then coupled with a classifier to form an end-to-end system that we apply to the large-scale sentiment classification task. SSE does not require feature selection, and its parameter space grows only linearly with the size of the n-grams.

READ MORE

Label Filters

In problems with a large number of labels, most multi-label and multi-class techniques incur a significant computational burden at test time. This is because, for each test instance, they need to systematically evaluate every label to decide whether it is relevant for the instance or not. We address this problem by designing computationally efficient label filters that eliminate the majority of labels from consideration before the base multi-class or multi-label classifier is applied. The proposed label filter projects a test instance on a filtering line and eliminates all the labels that had no training instances falling in the vicinity of this projection. The filter is learned directly from data by solving a constraint optimization problem, and it is independent of the base multi-label classifier. Experiments show that the proposed label filters can speed up prediction by orders of magnitude without significant impact on performance.

READ MORE