HorizonWeaver: Generalizable Multi-Level Semantic Editing for Driving Scenes

/in /by

Publication Date: 4/1/2026

Event: https://arxiv.org

Reference: https://arxiv.org/abs/2604.04887

Authors: Mauricio Soroco, Simon Fraser University; Francesco Pittaluga, NEC Laboratories America, Inc.; Zaid Tasneem, NEC Laboratories America, Inc.; Abhishek Aich, NEC Laboratories America, Inc.; Bingbing Zhuang, NEC Laboratories America, Inc.; Wuyang Chen, Simon Fraser University; Manmohan Chandraker, NEC Laboratories America, Inc.; Ziyu Jiang, NEC Laboratories America, Inc.

Abstract: Ensuring safety in autonomous driving requires scalable generation of realistic, controllable driving scenes beyond what real-world testing provides. Yet existing instruction guided image editors, trained on object-centric or artistic data, struggle with dense, safety-critical driving layouts. We propose HorizonWeaver, which tackles three fundamental challenges in driving scene editing: (1) multi-level granularity, requiring coherent object- and scene-level edits in dense environments; (2) rich high-level semantics, preserving diverse objects while following detailed instructions; and (3) ubiquitous domain shifts, handling changes in climate, layout, and traffic across unseen environments. The core of HorizonWeaver is a set of complementary contributions across data, model, and training: (1) Data: Large-scale dataset generation, where we build a paired real/synthetic dataset from Boreas, nuScenes, and Argoverse2 to improve generalization; (2) Model: Language-Guided Masks for fine-grained editing, where semantics-enriched masks and prompts enable precise, language-guided edits; and (3) Training: Content preservation and instruction alignment, where joint losses enforce scene consistency and instruction fidelity. Together, HorizonWeaver provides a scalable framework for photorealistic, instruction-driven editing of complex driving scenes, collecting 255K images across 13 editing categories and outperforming prior methods in L1, CLIP, and DINO metrics, achieving +46.4% user preference and improving BEV segmentation IoU by +33%. Project page: https://msoroco.github.io/horizonweaver/

Publication Link: https://arxiv.org/pdf/2604.04887