Entries by NEC Labs America

Radio-Frequency Linear Analysis and Optimization of Silicon Photonic Neural Networks

/in /by

Broadband analog signal processors utilizing silicon photonics have demonstrated a significant impact in numerous application spaces, offering unprecedented bandwidths, dynamic range, and tunability. In the past decade, microwave photonic techniques have been applied to neuromorphic processing, resulting in the development of novel photonic neural network architectures. Neuromorphic photonic systems can enable machine learning capabilities at extreme bandwidths and speeds. Herein, low-quality factor microring resonators are implemented to demonstrate broadband optical weighting. In addition, silicon photonic neural network architectures are critically evaluated, simulated, and optimized from a radio-frequency performance perspective. This analysis highlights the linear front-end of the photonic neural network, the effects of linear and nonlinear loss within silicon waveguides, and the impact of electrical preamplification.

Progressive Token Length Scaling in Transformer Encoders for Efficient Universal Segmentation

/in /by

A powerful architecture for universal segmentation relies on transformers that encode multi-scale image features and decode object queries into mask predictions. With efficiency being a high priority for scaling such models, we observed that the state-of-the-art method Mask2Former uses >50% of its compute only on the transformer encoder. This is due to the retention of a full-length token-level representation of all backbone feature scales at each encoder layer. With this observation, we propose a strategy termed PROgressive Token Length SCALing for Efficient transformer encoders (PRO-SCALE) that can be plugged-in to the Mask2Former style segmentation architectures to significantly reduce the computational cost. The underlying principle of PRO-SCALE is: progressively scale the length of the tokens with the layers of the encoder. This allows PRO-SCALE to reduce computations by a large margin with minimal sacrifice in performance (?52% GFLOPs reduction with no drop in performance on COCO dataset). We validate our frame work on multiple public benchmarks.

Efficient Transformer Encoders for Mask2Former-style Models

/in /by

Vision transformer based models bring significant improvements for image segmentation tasks. Although these architectures offer powerful capabilities irrespective of specific segmentation tasks, their use of computational resources can be taxing on deployed devices. One way to overcome this challenge is by adapting the computation level to the specific needs of the input image rather than the current one size-fits-all approach. To this end, we introduce ECO-M2F or EffiCient TransfOrmer Encoders for Mask2Former-style models. Noting that the encoder module of M2F-style models incur high resource-intensive computations, ECO-M2F provides a strategy to self-select the number of hidden layers in the encoder, conditioned on the input image. To enable this self-selection ability for providing a balance between performance and computational efficiency, we present a three-step recipe. The first step is to train the parent architecture to enable early exiting from the encoder. The second step is to create a derived dataset of the ideal number of encoder layers required for each training example. The third step is to use the aforementioned derived dataset to train a gating network that predicts the number of encoder layers to be used, conditioned on input image. Additionally, to change the computational-accuracy trade off, only steps two and three need to be repeated which significantly reduces retraining time. Experiments on the public datasets show that the proposed approach reduces expected encoder computational cost while maintaining performance, adapts to various user compute resources, is flexible in architecture configurations, and can be extended beyond the segmentation task to object detection.

Low-rank Constrained Multichannel Signal Denoising Considering Channel-dependent Sensitivity Inspired by Self-supervised Learning for Optical Fiber Sensing

/in /by

Optical fiber sensing is a technology wherein audio, vibrations, and temperature are detected using an optical fiber; especially the audio/vibrations-aware sensing is called distributed acoustic sensing (DAS). In DAS, observed data, which is comprised of multichannel data, has suffered from severe noise levels because of the optical noise or the installation methods. In conventional methods for denoising DAS data, signal-processing- or deep-neural-network (DNN)-based models have been studied. The signal-processing-based methods have the interpretability, i.e., non-black box. The DNN-based methods are good at flexibility designing network architectures and objective functions, that is, priors. However, there is no balance between the interpretability and the flexibility of priors in the DAS studies. The DNN-based methods also require a large amount of training data in general. To address the problems, we propose a DNN-structure signal-processing-based denoising method in this paper. As the priors of DAS, we employ spatial knowledge; low rank and channel-dependent sensitivity using the DNN-based structure.The result of fiber-acoustic sensing shows that the proposed method outperforms the conventional methods and the robustness to the number of the spatial ranks. Moreover, the optimized parameters of the proposed method indicate the relationship with the channel sensitivity; the interpretability.

Provable Membership Inference Privacy

/in /by

In applications involving sensitive data, such as finance and healthcare, the necessity for preserving data privacy can be a significant barrier to machine learning model development.Differential privacy (DP) has emerged as one canonical standard for provable privacy. However, DP’s strong theoretical guarantees often come at the cost of a large drop in its utility for machine learning; and DP guarantees themselves are difficult to interpret. In this work, we propose a novel privacy notion, membership inference privacy (MIP), as a steptowards addressing these challenges. We give a precise characterization of the relationship between MIP and DP, and show that in some cases, MIP can be achieved using less amountof randomness compared to the amount required for guaranteeing DP, leading to smaller drop in utility. MIP guarantees are also easily interpretable in terms of the success rate of membership inference attacks in a simple random subsampling setting. As a proof of concept, we also provide a simple algorithm for guaranteeing MIP without needing to guarantee DP.

Link Loss Analysis of Integrated Linear Weight Bank within Silicon Photonic Neural Network

/in /by

Over the last decade, silicon photonic neural networks have demonstrated the possibility of photonic-enabled machine learning at the edge. These systems enable low-latency ultra-wideband classifications, channel estimations, and many other signal characterization tasks within wireless environments. While these proof-of-concept experiments have yielded promising results, poor device and architectural designs have resulted in sub-optimal bandwidth and noise performance. As a result, the application space of this technology has been limited to GHz bandwidths and high signal-to-ratio input signals. By applying a microwave photonic perspective to these systems, the authors demonstrate high-bandwidth operation while optimizing for RF performance metrics: instantaneous bandwidth, link loss, noise figure, and dynamic range. The authors explore the extended capabilities due to these improved metrics and potential architectures to continue further optimization. The authors introduce novel architectures and RF analysis for RF-optimized neuromorphic photonic hardware.

NEC Labs America at OFC 2024 San Diego from March 24 – 28

/in /by

The NEC Labs America team Yaowen Li, Andrea D’Amico, Yue-Kai Huang, Philip Ji, Giacomo Borraccini, Ming-Fang Huang, Ezra Ip, Ting Wang & Yue Tian (Not pictured: Fatih Yaman) has arrived in San Diego, CA for OFC24! Our team will be speaking and presenting throughout the event. Read more for an overview of our participation.