StreetAware: A High-Resolution Synchronized Multimodal Urban Scene Dataset

/in /by

Access to high-quality data is an important barrier in the digital analysis of urban settings, including applications within computer vision and urban design. Diverse forms of data collected from sensors in areas of high activity in the urban environment, particularly at street intersections, are valuable resources for researchers interpreting the dynamics between vehicles, pedestrians, and the built environment. In this paper, we present a high-resolution audio, video, and LiDAR dataset of three urban intersections in Brooklyn, New York, totaling almost 8 unique hours. The data were collected with custom Reconfigurable Environmental Intelligence Platform (REIP) sensors that were designed with the ability to accurately synchronize multiple video and audio inputs. The resulting data are novel in that they are inclusively multimodal, multi-angular, high-resolution, and synchronized. We demonstrate four ways the data could be utilized — (1) to discover and locate occluded objects using multiple sensors and modalities, (2) to associate audio events with their respective visual representations using both video and audio modes, (3) to track the amount of each type of object in a scene over time, and (4) to measure pedestrian speed using multiple synchronized camera views. In addition to these use cases, our data are available for other researchers to carry out analyses related to applying machine learning to understanding the urban environment (in which existing datasets may be inadequate), such as pedestrian-vehicle interaction modeling and pedestrian attribute recognition. Such analyses can help inform decisions made in the context of urban sensing and smart cities, including accessibility-aware urban design and Vision Zero initiatives.

Read more

Explore Benefits of Distributed Fiber Optic Sensing for Optical Network Service Providers

/in /by

We review various applications of distributed fiber optic sensing (DFOS) and machine learning (ML) technologies that particularly benefit telecom operators’ fiber networks and businesses. By leveraging relative phase shift of the reflectance of coherent Rayleigh, Brillouin and Raman scattering of light wave, the ambient environmental vibration, acoustic effects, temperature and fiber/cable strain can be detected. Fiber optic sensing technology allows optical fiber to support sensing features in addition to its conventional role to transmit data in telecommunications. DFOS has recently helped telecom operators by adding multiple sensing features and proved feasibility of co-existence of sensing and communication systems on same fiber. We review the architecture of DFOS technique and show examples where optical fiber sensing helps enhance network operation efficiency and create new services for customers on deployed fiber infrastructures, such as determination of cable locations, cable cut prevention, perimeter intrusion detection and networked sensing applications. In addition, edge AI platform allows data processing to be conducted on-the-fly with low latency. Based on discriminative spatial-temporal signatures of different events of interest, real-time processing of the sensing data from the DFOS system provides results of the detection, classification and localization immediately.

Read more

RIS-aided mmWave Beamforming for Two-way Communications of Multiple Pairs

/in /by

Millimeter‑wave (mmWave) communications is a key enabler towards realizing enhanced Mobile Broadband (eMBB) as a key promise of 5G and beyond, due to the abundance of bandwidth available at mmWave bands. An mmWave coverage map consists of blind spots due to shadowing and fading especially in dense urban environments. Beamformingemploying massive MIMO is primarily used to address high attenuation in the mmWave channel. Due to their ability in manipulating the impinging electromagnetic waves in an energy‑efficient fashion, Reconfigurable Intelligent Surfaces (RISs) are considered a great match to complement the massive MIMO systems in realizing the beamforming task and therefore effectively filling in the mmWave coverage gap. In this paper, we propose a novel RIS architecture, namely RIS‑UPA where the RIS elements are arranged in a Uniform Planar Array (UPA). We show how RIS‑UPA can be used in an RIS‑aided MIMO system to fill the coverage gap in mmWave by forming beams of a custom footprint, with optimized main lobe gain, minimum leakage, and fairly sharp edges. Further, we propose a configuration for RIS‑UPA that can support multiple two‑way communication pairs, simultaneously. We theoretically obtain closed‑form low‑complexity solutions for our design and validate our theoretical findings by extensive numerical experiments.

Read more

Channel Reciprocity Calibration for Hybrid Beamforming in Distributed MIMO Systems

/in /by

Time Division Duplex (TDD)-based distributed massive MIMO systems are envisioned as candidate solution for the physical layer of 6G multi-antenna systems supporting cooperative hybrid beamforming that heavily relies on the obtained uplink channel estimates for efficient coherent downlink precoding. However, due to the hardware impairment between the transmitter and the receiver, full channel reciprocity does not hold between the downlink and uplink direction. Such reciprocity mismatch deteriorates the performance of mm-Wave hybrid beamforming and has to be estimated and compensated for, to avoid performance degradation in the co-operative hybrid beamforming. In this paper, we address the channel reciprocity calibration between any two nodes at two levels. We decompose the problem into two sub-problems. In the first sub-problem, we calibrate the digital chain, i.e. obtain the mismatch coefficients of the (DAC/ADC) up to a constant scaling factor. In the second subproblem, we obtain the (PA/LNA) mismatch coefficients. At each step, we formulate the channel reciprocity calibration as a least square optimization problem that can efficiently be solved via conventional methods such as alternative optimization with high accuracy. Finally, we verify the performance of our channel reciprocity calibration approach through extensive numerical experiments.

Read more

Interpretable Skill Learning for Dynamic Treatment Regimes through Imitation

/in /by

Imitation learning that mimics experts’ skills from their demonstrations has shown great success in discovering dynamic treatment regimes, i.e., the optimal decision rules to treat an individual patient based on related evolving treatment and covariate history. Existing imitation learning methods, however, still lack the capability to interpret the underlying rationales of the learned policy in a faithful way. Moreover, since dynamic treatment regimes for patients often exhibit varying patterns, i.e., symptoms that transit from one to another, the flat policy learned by a vanilla imitation learning method is typically undesired. To this end, we propose an Interpretable Skill Learning (ISL) framework to resolve the aforementioned challenges for dynamic treatment regimes through imitation. The key idea is to model each segment of experts’ demonstrations with a prototype layer and integrate it with the imitation learning layer to enhance the interpretation capability. On one hand, the ISL framework is able to provide interpretable explanations by matching the prototype to exemplar segments during the inference stage, which enables doctors to perform reasoning of the learned demonstrations based on human-understandable patient symptoms and lab results. On the other hand, the obtained skill embedding consisting of prototypes serves as conditional information to the imitation learning layer, which implicitly guides the policy network to provide a more accurate demonstration when the patients’ state switches from one stage to another. Thoroughly empirical studies demonstrate that our proposed ISL technique can achieve better performance than state-of-the-art methods. Moreover, the proposed ISL framework also exhibits good interpretability which cannot be observed in existing methods.

Read more

Data-driven Modelling of EDFAs by Neural Networks

/in /by

Dependence of EDFA gain shape on input power and input spectrum shape is modelled using a simple neural network-based architecture for amplifiers with different gains and output powers. The model can predict the gain within ±0.1 dB. Even though the model has good success predicting the performance of the particular EDFA it is trained with, it is not as successful when used to predict a different EDFA, or even the same EDFA with a different pump power. However, retraining the model with a small amount of supplementary data from a second EDFA makes the model able to predict the performance of the second EDFA with little loss in performance.

Read more

Improvement of Resilience of Submarine Networks Based on Fiber Sensing

/in /by

Simultaneous phase and polarization sensing with span length resolution using the supervisory path is demonstrated. It is shown that by measuring polarization rotation matrix of the return paths, instead of monitoring only the state of polarization, location of the polarization disturbance can be determined even for large polarization rotations. By using the polarization rotation matrices, the phase and polarization disturbances are successfully decoupled. How the existing supervisory system and sensing can coexist in new SDM cables that utilizes pump sharing is discussed.

Read more

Dynamic Prompting: A Unified Framework for Prompt Tuning

/in /by

It has been demonstrated that prompt tuning is highly effective in efficiently eliciting knowledge from language models (LMs). However, the prompt tuning still lags behind fine tuning, especially when the LMs are small. P tuning v2 (Liu et al., 2021b) makes it comparable with finetuning by adding continuous prompts for every layer of the pre trained model. However, prepending fixed soft prompts for all instances, regardless of their discrepancy, is doubtful. In particular, the inserted prompt position, length, and the representations ofprompts for diversified instances through different tasks could all affect the prompt tuning performance. To fill this gap, we propose dynamic prompting (DP): the position, length, and prompt representation can all be dynamically optimized with respect to different tasks and instances. We conduct comprehensive experiments on the SuperGlue benchmark tovalidate our hypothesis and demonstrate substantial improvements. We also derive a unified framework for supporting our dynamic prompting strategy. In particular, we use a simple learning network and Gumble Softmax for learning instance dependent guidance. Experimental results show that simple instance level position aware soft prompts can improve the classification accuracy of up to 6 points on average on five datasets, reducing its gap with fine tuning. Besides, we also prove its universal usefulness under full data, few shot, andmultitask regimes. Combining them together can even further unleash the power of DP, narrowing the distance between fine tuning.

Read more

Field Trial of Coexistence and Simultaneous Switching of Real-time Fiber Sensing and 400GbE Supporting DCI and 5G Mobile Services

/in /by

Coexistence of real-time constant-amplitude distributed acoustic sensing (DAS) and 400GbE signals is verified by field trial over metro fibers, demonstrating no QoT impact during co-propagation and supporting preemptive DAS-informed optical path switching before link failure

Read more

Polarization Sensing Using Polarization Rotation Matrix Eigenvalue Method

/in /by

Polarization-based, multi-span sensing over a link with reflection-back circuits is demonstrated experimentally. By measuring rotation matrices instead of just monitoring polarization, a 35 dB extinction in localization is achieved regardless of the disturbance magnitude.

Read more