Blog

Welcome to the Media Analytics Department's blog, where we cover important topics related to computer vision and machine learning in a more informal medium than a research paper.


01-06-21 | Universal Representations for Responsible and Robust AI

Responsible AI systems for surveillance, healthcare, and biometrics applications that focus on people need to satisfy legal and social requirements. Moreover, robust AI engines need to achieve accuracy in new and unexpected situations for applications such as mobility, insurance, construction, and factories. To enable both responsible and robust AI systems while reducing cost of data collection and annotation, we present a URLearning framework to learn universal representations for visual analysis of scenes, objects and people in varied situations. e.g., weather, lighting, and ethnicity. By leveraging both labeled and unlabeled data with advanced techniques such as domain generalization, domain adaptation, and semi-supervised leanring, we demonstrate diverse computer vision use cases for surveillance and mobility applications.

Read the post...
01-06-21 | Privacy-Aware AI Models

We develop two types of technologies for privacy-aware AI models: differential privacy and differentially private federated Learning. These technologies prevent attackers from using model outputs to decipher data, either in stand-alone training or in combined training across multiple stakeholders. Furthermore, Our privacy-aware AI models achieve high privacy and accuracy, with less data required compared to competitors. For example, Google needs much larger data than us, so cannot achieve high accuracy when customer data is limited. Other approaches such as federated learning used by Apple achieve accuracy, but do not provide a provable guarantee of privacy. As such, our models are suitable for a range of applications such as surveillance with face and human data, as well as finance with data from multiple banks.

Read the post...
9-24-20 | Understanding Road Scenes in Videos

Given a video of a complex scene, we represent various elements of the scene in a way that is both interpretable and accurate. This allows explainable decision-making for downstream application and intuitive visualization for human-machine interaction. Specifically, for a road scene, we predict elements of the layout such as the number of lanes, positions of crosswalks, distance from the intersection and so on. These predictions are constrained to be consistent across the frames of the video and also incorporate contextual cues such as positions of other objects in the scene. We achieve such consistency across time, egomotion and object states through novel deep neural architectures, which leads to state-of-the-art performances on challenging public benchmarks.

Read the post...
09-17-20 | AI Solutions with a Guarantee of Privacy

With increasing ethical and legal concerns on privacy for deep models in visual recognition, differential privacy has emerged as a mechanism to disguise membership of sensitive data in training datasets. Recent methods like Private Aggregation of Teacher Ensembles (PATE) leverage a large ensemble of teacher models trained on disjoint subsets of private data, to transfer knowledge to a student model with privacy guarantees. However, labeled vision data is often expensive and datasets when split into many disjoint training sets lead to significantly sub-optimal accuracy and thus hardly sustain good privacy bounds. We propose a practically data-efficient scheme based on private release of k-nearest neighbor (kNN) queries, which altogether avoids splitting the training dataset.

Read the post...
09-10-20 | Media Analytics Research at CVPR 2020

CVPR is a highly rated conference for computer vision, for which a virtual version was held in June 2020. The MA team from NEC Labs America presented four papers at the conference. These papers covered a range of topics including outdoor and indoor 3D scene understanding, learning universal representations for face recognition in the wild and practical methods for differential privacy that achieve high levels of theoretical guarantee and accuracy even with limited data.

Read the post...