On Generalizing Beyond Domains in Cross-Domain Continual Learning

/in /by

Humans have the ability to accumulate knowledge of new tasks in varying conditions, but deep neural networks of-ten suffer from catastrophic forgetting of previously learned knowledge after learning a new task. Many recent methods focus on preventing catastrophic forgetting under the assumption of train and test data following similar distributions. In this work, we consider a more realistic scenario of continual learning under domain shifts where the model must generalize its inference to an unseen domain. To this end, we encourage learning semantically meaningful features by equipping the classifier with class similarity metrics as learning parameters which are obtained through Mahalanobis similarity computations. Learning of the backbone representation along with these extra parameters is done seamlessly in an end-to-end manner. In addition, we propose an approach based on the exponential moving average of the parameters for better knowledge distillation. We demonstrate that, to a great extent, existing continual learning algorithms fail to handle the forgetting issue under multiple distributions, while our proposed approach learns new tasks under domain shift with accuracy boosts up to 10% on challenging datasets such as DomainNet and OfficeHome.

Read more

MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation

/in /by

Test-time adaptation approaches have recently emerged as a practical solution for handling domain shift without access to the source domain data. In this paper, we propose and explore a new multi-modal extension of test-time adaptation for 3D semantic segmentation. We find that, directly applying existing methods usually results in performance instability at test time, because multi-modal input is not considered jointly. To design a framework that can take full advantage of multi-modality, where each modality provides regularized self-supervisory signals to other modalities, we propose two complementary modules within and across the modalities. First, Intra-modal Pseudo-label Generation (Intra-PG) is introduced to obtain reliable pseudo labels within each modality by aggregating information from two models that are both pre-trained on source data but updated with target data at different paces. Second, Inter-modal Pseudo-label Refinement (Inter-PR) adaptively selects more reliable pseudo labels from different modalities based on a proposed consistency scheme. Experiments demonstrate that our regularized pseudo labels produce stable self-learning signals in numerous multi-modal test-time adaptation scenarios for 3D semantic segmentation. Visit our project website at https://www.nec-labs.com/~mas/MM-TTA

Read more

Learning to Learn across Diverse Data Biases in Deep Face Recognition

/in /by

Convolutional Neural Networks have achieved remarkable success in face recognition, in part due to the abundant availability of data. However, the data used for training CNNs is often imbalanced. Prior works largely focus on the long-tailed nature of face datasets in data volume per identity or focus on single bias variation. In this paper, we show that many bias variations such as ethnicity, head pose, occlusion and blur can jointly affect the accuracy significantly. We propose a sample level weighting approach termed Multi-variation Cosine Margin (MvCoM), to simultaneously consider the multiple variation factors, which orthogonally enhances the face recognition losses to incorporate the importance of training samples. Further, we leverage a learning to learn approach, guided by a held-out meta learning set and use an additive modeling to predict the MvCoM. Extensive experiments on challenging face recognition benchmarks demonstrate the advantages of our method in jointly handling imbalances due to multiple variations.

Read more

Controllable Dynamic Multi-Task Architectures

/in /by

Multi-task learning commonly encounters competition for resources among tasks, specifically when model capacity is limited. This challenge motivates models which allow control over the relative importance of tasks and total compute cost during inference time. In this work, we propose such a controllable multi-task network that dynamically adjusts its architecture and weights to match the desired task preference as well as the resource constraints. In contrast to the existing dynamic multi-task approaches that adjust only the weights within a fixed architecture, our approach affords the flexibility to dynamically control the total computational cost and match the user-preferred task importance better. We propose a disentangled training of two hyper networks, by exploiting task affinity and a novel branching regularized loss, to take input preferences and accordingly predict tree-structured models with adapted weights. Experiments on three multi-task benchmarks, namely PASCAL-Context, NYU-v2, and CIFAR-100, show the efficacy of our approach. Project page is available at https://www.nec-labs.com/-mas/DYMU.

Read more

Distributed Fiber Optic Sensors Placement for Infrastructure-as-a-Sensor

/in /by

Recently, the distributed fiber optic sensing (DFOS) techniques have advanced rapidly. There emerges various types of DFOS sensors that can monitor physical parameters such as temperature, strain, and vibration. With these DFOS sensors deployed, the telecom networks are capable of offering additional services beyond communications, such as monitoring road traffic condition, monitoring utility pole health, monitoring city noise and accident, thus evolving to a new paradigm of Infrastructure-as-a-Sensor (IaaSr) or Network-as-a-Sensor (NaaSr). When telecom network carriers upgrade their infrastructures with DFOS sensors to provide such IaaSr/NaaSr services, there will arise a series of critical challenges: (1) where to place the DFOS sensors, and (2) how to provision the DFOS sensing fiber routes to cover the whole network infrastructures with the minimum number of DFOS sensors? We name this as the DFOS placement problem. In this paper, we prove that the DFOS placement problem is an NP-hard problem, and we analyze the upper bound of the number of DFOS sensors used. To facilitate the optimal solution, we formulate the DFOS placement problem with an Integer Linear Programming model that aims at minimizing the number of DFOS sensors used. Furthermore, we propose a cost-efficient heuristic solution, called Explore-and-Pick (EnP), which can achieve a close-to-optimal performance in a fast manner. We analyze the approximation ratio and the computational complexity of the proposed EnP algorithm. In addition, we conduct comprehensive simulations to evaluate the performance of the proposed solutions. Simulation results show that the EnP algorithm can outperform the baseline algorithm by 16% in average and 26% at best, and it achieves a performance that is close to the optimal result obtained by ILP.

Read more

SEED: Sound Event Early Detection via Evidential Uncertainty

/in /by

Sound Event Early Detection (SEED) is an essential task in recognizing the acoustic environments and soundscapes. However, most of the existing methods focus on the offline sound event detection, which suffers from the over-confidence issue of early-stage event detection and usually yield unreliable results. To solve the problem, we propose a novel Polyphonic Evidential Neural Network (PENet) to model the evidential uncertainty of the class probability with Beta distribution. Specifically, we use a Beta distribution to model the distribution of class probabilities, and the evidential uncertainty enriches uncertainty representation with evidence information, which plays a central role in reliable prediction. To further improve the event detection performance, we design the backtrack inference method that utilizes both the forward and backward audio features of an ongoing event. Experiments on the DESED database show that the proposed method can simultaneously improve 13.0% and 3.8% in time delay and detection F1 score compared to the state-of-the-art methods.

Read more

Fast Few-shot Debugging for NLU Test Suites

/in /by

We study few-shot debugging of transformer based natural language understanding models, using recently popularized test suites to not just diagnose but correct a problem. Given a few debugging examples of a certain phenomenon, and a held-out test set of the same phenomenon, we aim to maximize accuracy on the phenomenon at a minimal cost of accuracy on the original test set. We examine several methods that are faster than full epoch retraining. We introduce a new fast method, which samples a few in-danger examples from the original training set. Compared to fast methods using parameter distance constraints or Kullback-Leibler divergence, we achieve superior original accuracy for comparable debugging accuracy.

Read more

NEC provides AI-based traffic monitoring system with fiber-optic sensing technology for NEXCO CENTRAL

/in /by

NEC Corporation has deployed an AI-based traffic monitoring system to Central Nippon Expressway Company Limited (NEXCO CENTRAL). The system uses fiber-optic sensing and AI technologies to visualize traffic conditions, such as the location, speed, and direction of travel, from vibrations produced by vehicle movement.

Read more

Codebook Design for Hybrid Beamforming in 5G Systems

/in /by

Massive MIMO and hybrid beamforming are among the key physical layer technologies for the next generation wireless systems. In the last stage of the hybrid beamforming, the goal is to generate sharp beam with maximal and preferably uniform gain. We highlight the shortcomings of uniform linear arrays (ULAs) in generating such perfect beams, i.e., beams with maximal uniform gain and sharp edges, and propose a solution based on a novel antenna configuration, namely, twin-ULA (TULA). Consequently, we propose two antenna configurations based on TULA: Delta and Star. We pose the problem of finding the beamforming coefficients as a continuous optimization problem for which we find the analytical closed-form solution by a quantization/aggregation method. Thanks to the derived closed-form solution the beamforming coefficients can be easily obtained with low complexity. Through numerical analysis, we illustrate the effectiveness of the proposed antenna structure and beamforming algorithm to reach close-to-perfect beams.

Read more

Time Series Prediction and Classification using Silicon Photonic Neuron with Self-Connection

/in /by

We experimentally demonstrated the real-time operation of a photonic neuron with a self-connection, a prerequisite for integrated recurrent neural networks (RNNs). After studying two applications, we propose a photonics-assisted platform for time series prediction and classification.

Read more