Which Should We Test Next? Performance Gap Discovery for Driving VLMs

/in /by

Publication Date: 5/27/2026

Event: https://arxiv.org

Reference: pp. 1-23

Authors: Abhishek Aich, NEC Laboratories America, Inc.; Sparsh Garg, NEC Laboratories America, Inc.; Vijay Kumar, NEC Laboratories America, Inc.; Turgun Kashgari, NEC Laboratories America, Inc.; Manmohan Chandraker, NEC Laboratories America, Inc.

Abstract: Driving vision-language models (VLMs) must accurately understand scenes across diverse conditions defined by Operational Design Domains (ODDs), yet verification remains sparse: many slices are missing, making empirical failure rates unreliable. We propose SLICESCORER, a deterministic scoring rule for missing-slicerecommendation that combines (i) an exposure-based coverage prior to prioritize rare, under-tested regions, and (ii) a neighbor-failure prior that propagates risk from similar tested conditions. SLICESCORER is deliberately simple- interpretable, auditable, and conservative- properties essential for safety-criticalvalidation. For stress testing beyond the declared ODD, we embed SLICESCORER within SLICENAV, an LLM-orchestrated verification pipeline where the model interprets developer queries to select relevant operators (triage, scoring, acquisition, evaluation) and vocabulary extensions, composing verification workflows while keeping all scoring deterministic and auditable. Experiments on three driving VLMs(WiseAD, DriveMM, Cosmos-Reason2-2B) show that SLICENAV surfaceshigh-risk coverage gaps more effectively than prior slice-discovery methods while maintaining diverse recommendations across the condition space. Ablations confirm both scoring components contribute, and qualitative analysis demonstrates end-to-end workflows from developer query to targeted evaluation.

Publication Link: https://arxiv.org/pdf/2606.01624