Posts

RoVaR: Robust Multi-agent Tracking through Dual-layer Diversity in Visual and RF Sensor Fusion

/in /by

RoVaR: Robust Multi-agent Tracking through Dual-layer Diversity in Visual and RF Sensor Fusion The plethora of sensors in our commodity devices provides a rich substrate for sensor-fused tracking. Yet, today’s solutions are unable to deliver robust and high tracking accuracies across multiple agents in practical, everyday environments – a feature central to the future of immersive and collaborative applications. This can be attributed to the limited scope of diversity leveraged by these fusion solutions, preventing them from catering to the multiple dimensions of accuracy, robustness (diverse environmental conditions) and scalability (multiple agents) simultaneously.In this work, we take an important step towards this goal by introducing the notion of dual-layer diversity to the problem of sensor fusion in multi-agent tracking. We demonstrate that the fusion of complementary tracking modalities, – passive/relative (e.g. visual odometry) and active/absolute tracking (e.g.infrastructure-assisted RF localization) offer a key first layer of diversity that brings scalability while the second layer of diversity lies in the methodology of fusion, where we bring together the complementary strengths of algorithmic (for robustness) and data-driven (for accuracy) approaches. ROVAR is an embodiment of such a dual-layer diversity approach that intelligently attends to cross-modal information using algorithmic and data-driven techniques that jointly share the burden of accurately tracking multiple agents in the wild. Extensive evaluations reveal ROVAR’S multi-dimensional benefits in terms of tracking accuracy, scalability and robustness to enable practical multi-agent immersive applications in everyday environments.

Read more

RoVaR: Robust Multi agent Tracking through Dual layer Diversity in Visual and RF Sensor Fusion

/in /by

RoVaR: Robust Multi agent Tracking through Dual layer Diversity in Visual and RF Sensor Fusion The plethora of sensors in our commodity devices provides a rich substrate for sensor fused tracking. Yet, today’s solutions are unable to deliver robust and high tracking accuracies across multiple agents in practical, everyday environments a feature central to the future of immersive and collaborative applications. This can be attributed to the limited scope of diversity leveraged by these fusion solutions, preventing them from catering to the multiple dimensions of accuracy, robustness (diverse environmental conditions) and scalability (multiple agents) simultaneously. In this work, we take an important step towards this goal by introducing the notion of dual layer diversity to the problem of sensor fusion in multi agent tracking. We demonstrate that the fusion of complementary tracking modalities, passive/relative (e.g., visual odometry) and active/absolute tracking (e.g., infrastructure assisted RF localization) offer a key first layer of diversity that brings scalability while the second layer of diversity lies in the methodology of fusion, where we bring together the complementary strengths of algorithmic (for robustness) and data driven (for accuracy) approaches. RoVaR is an embodiment of such a dual layer diversity approach that intelligently attends to cross modal information using algorithmic and data driven techniques that jointly share the burden of accurately tracking multiple agents in the wild. Extensive evaluations reveal RoVaR’s multi dimensional benefits in terms of tracking accuracy (median of 15cm), robustness (in unseen environments), light weight (runs in real time on mobile platforms such as Jetson Nano/TX2), to enable practical multi agent immersive applications in everyday environments.

Read more

SkyHAUL: A Self-Organizing Gigabit Network In The Sky

/in /by

SkyHAUL: A Self-Organizing Gigabit Network In The Sky We design and build SkyHaul, the first large-scale, self-organizing network of Unmanned Aerial Vehicles (UAVs) that are connected using a mm Wave wireless mesh backhaul. While the use of a mmWave backhaul paves the way for a new class of bandwidth-intensive, latency-sensitive cooperative applications (e.g. LTE coverage during disasters), the network of UAVs allows these applications to be executed at operating ranges that are far beyond the line-of-sight distances that limit individual UAVs today.To realize the challenging vision of deploying and maintaining an airborne, mm Wave mesh backhaul that caters to dynamic applications, SkyHaul’s design incorporates various elements: (i) Role-specific UAV operations that simultaneously address application tracking and backhaul connectivity (ii) Novel algorithms to jointly address the problem of deployment (position, yaw of UAVs) and traffic routing across the UAV network, and (iii)A provably optimal solution for fast and safe reconfiguration of UAV backhaul during application dynamics. We evaluate the performance of SkyHaul through both real-world UAV flight operations as well as large scale simulations.

Read more

SkyHaul: An Autonomous Gigabit Network Fabric In The Sky

/in /by

SkyHaul: An Autonomous Gigabit Network Fabric In The Sky We design and build SKYHAUL, the first large scale, autonomous, self organizing network of Unmanned Aerial Vehicles (UAVs) that are connected using a mmWave wireless mesh backhaul. While the use of a mmWave backhaul paves the way for a new class of bandwidth intensive, latency sensitive cooperative applications (e.g., LTE coverage during disasters, surveillance during rescue in challenging terrains), the network of UAVs allows these applications to be executed at operating ranges that are far beyond the line of sight distances that limit individual UAVs today. To realize the challenging vision of deploying and maintaining an airborne mmWave mesh backhaul to cater to dynamic applications, SKYHAUL’s design incorporates various elements: (1) Role specific UAV operations that simultaneously address application tracking and backhaul connectivity (2) Novel algorithms to jointly address the problem of deployment (position, yaw of UAVs) and traffic routing across the UAV network, and (3) A provably optimal solution for fast and safe reconfiguration of UAV backhaul during application dynamics. We implement SKYHAUL on four DJI Matrice 600 Pros to demonstrate its practicality and performance through autonomous flight operations, complemented by large scale simulations.

Read more