Publication Date: 9/8/2026

Event: 6th DriveX Workshop In conjunction with ECCV 2026 , Malmö, Sweden

Reference: pp. 1-18, 2026

Authors: Yi-Wen Chen, NEC Laboratories America, Inc.; Vijay Kumar B G, NEC Laboratories America, Inc.; Sparsh Garg, NEC Laboratories America, Inc.; Manmohan Chandraker, NEC Laboratories America, Inc., UC San Diego

Abstract: Vision-language models (VLMs) have demonstrated strong capabilities in visual understanding and multimodal reasoning, yet adapting them to autonomous driving remains challenging due to the dynamic, safety-critical nature of driving environments. In particular, existing open-source VLMs often struggle to reason about hypothetical events and their potential consequences, limiting their usefulness for driving-related decision support. We present CounterDrive, a three-stage adaptation framework that enhances driving-domain counterfactual and spatiotemporal reasoning in open-source VLMs. First, we perform domain pre-alignment using mobility-oriented supervision to adapt pretrained models to driving scenes and motion dynamics. Second, we conduct mobility QA finetuning with a mixture of factual driving questions and counterfactual question-answer pairs, enabling the model to connect scene understanding with reasoning about alternative outcomes. Third, we apply preference optimization to encourage grounded, coherent, and safety-oriented responses in challenging driving scenarios. The framework is scalable and leverages automatically constructed supervision, substantially reducing the need for manual annotation. Experiments on MMERealWorld, DriveBench, and a dedicated counterfactual driving benchmark demonstrate consistent improvements over InternVL3 baselines at both 1B and 8B scales, while maintaining competitive performance on general-domain multimodal benchmarks. These results show that targeted adaptation can effectively transfer driving-domain counterfactual reasoning capabilities to compact open-source VLMs.

Publication Link: