LangDriveCTRL: Natural Language Controllable Driving Scene Editing with Multi-modal Agents

/in /by

LangDriveCTRL is a natural-language-controllable framework for editing real-world driving videos to synthesize diverse traffic scenarios. It represents each video as an explicit 3D scene graph, decomposing the scene into a static background and dynamic object nodes. To enable fine-grained editing and realism, it introduces a feedback-driven agentic pipeline. An Orchestrator converts user instructions into executable graphs that coordinate specialized multi-modal agents and tools. An Object Grounding Agent aligns free-form text with target object nodes in the scene graph; a Behavior Editing Agent generates multi-object trajectories from language instructions; and a Behavior Reviewer Agent iteratively reviews and refines the generated trajectories. The edited scene graph is rendered and harmonized using a video diffusion tool, and then further refined by a Video Reviewer Agent to ensure photorealism and appearance alignment. LangDriveCTRL supports both object node editing (removal, insertion, and replacement) and multi-object behavior editing from natural-language instructions. Quantitatively, it achieves nearly higher instruction alignment than the previous SoTA, with superior photorealism, structural preservation, and traffic realism.

Read more

PhyCo: Learning Controllable Physical Priors for Generative Motion

/in /by

Modern video diffusion models excel at appearance synthesis but still struggle with physical consistency: objects drift, collisions lack realistic rebound, and material responses seldom match their underlying properties. We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation. Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback. This combination enables a generative model to produce physically consistent and controllable outputs through variations in physical attributes–without any simulator or geometry reconstruction at inference. On the Physics-IQ benchmark, PhyCo significantly improves physical realism over strong baselines, and human studies confirm clearer and more faithful control over physical attributes. Our results demonstrate a scalable path toward physically consistent, controllable generative video models that generalize beyond synthetic training environments.

Read more

Uncertainty-Aware Knowledge Distillation for Multimodal Large Language Models

/in /by

Knowledge distillation establishes a learning paradigm that leverages both data supervision and teacher guidance. However, determining the optimal balance between learning from data and learning from the teacher is challenging, as some samples may be noisy while others are subject to teacher uncertainty. This motivates the need for adaptively balancing data and teacher supervision. We propose Beta-weighted Knowledge Distillation (Beta-KD), an uncertainty-aware distillation framework that adaptively modulates how much the student relies on teacher guidance. Specifically, we formulate teacher-student learning from a unified Bayesian perspective and interpret teacher supervision as a Gibbs prior over student activations. This yields a closed-form, uncertainty-aware weighting mechanism and supports arbitrary distillation objectives and their combinations. Extensive experiments are conducted on multimodal VQA benchmarks by distilling a student Vision-Language Model from a large teacher VLM. The results demonstrate that Beta-KD consistently outperforms existing knowledge distillation methods. Code is available at https://github.com/Jingchensun/beta-kd.

Read more

Exact PDL-Induced SNR Statistics in Dual-Polarization Coherent Optical Communication Systems

/in /by

We derive the exact probability density function of the signal to noise ratio (SNR) impairment caused by polarization dependent loss (PDL) in coherent optical systems. Using thehinge model for PDL statistics, we construct a sequence of variable transformations that yields the expression of the SNR probability density function by means of a sequence of nested recursive integrals. Monte Carlo simulations validate the analytical formulation assuming realistic PDL levels and link configurations, showing excellent agreement. The model accurately captures SNR variability introduced by cascaded PDL elements and provides a foundation for statistical margin assessment, digital twin modeling, and quality of transmission estimation in flexible, low marginoptical networks.

Read more

Automated Negotiation and Multimodal Time-Series Forecasting for Efficient Procurement

/in /by

Procurement is a key function in supply chain management that involves acquiring goods and services to meet organizational needs. Efficient procurement is crucial for minimizing costs, ensuring timely delivery, and maintaining quality standards. This paper explores the integration of automated negotiation and multimodal time-series forecasting to enhance procurement processes. Automated negotiation can streamline interactions with suppliers, while multimodal time-series forecasting can improve demand prediction accuracy by leveraging diverse data sources leading to better negotiation outputs. By combining these approaches, organizations can optimize procurement strategies, reduce costs, and improve overall supply chain efficiency. We present two case studies using simulations based on real-world data for procurement that show the effectiveness of the proposed framework.

Read more

Driving Video Retrieval for Complex Queries with Structured Grounding

/in /by

Video retrieval at scale is central to data curation and safety validation in autonomous driving, where users want to find not only scenes but also dynamic events such as cut-ins and hard braking. Existing vision-language and keyword-based retrieval methods often miss these events because the relevant motion may not be explicitly described in text or captured by lexical overlap. Rule-based retrieval can encode such events more directly, but it is brittle: generated or hand-written rules often fail when their assumptions do not match real driving data. We propose STRIVE-D, a data-calibrated retrieval framework for driving videos. It uses weakly labeled in-domain videos to estimate when a query rule is reliable, adapt rules that mismatch observed data, and fuse calibrated rule scores with vision-language and keyword-based retrieval signals. Across three driving benchmarks, including newly released human-annotated event data on DrivingDojo, STRIVE-D delivers up to 84% relative improvement in top-1 accuracy over state-of-the-art methods.

Read more

Which Should We Test Next? Performance Gap Discovery for Driving VLMs

/in /by

Driving vision-language models (VLMs) must accurately understand scenes across diverse conditions defined by Operational Design Domains (ODDs), yet verificationremains sparse: many slices are missing, making empirical failure rates unreliable. We propose SLICESCORER, a deterministic scoring rule for missing-slicerecommendation that combines (i) an exposure-based coverage prior to prioritize rare, under-tested regions, and (ii) a neighbor-failure prior that propagatesrisk from similar tested conditions. SLICESCORER is deliberately simple- interpretable, auditable, and conservative- properties essential for safety-criticalvalidation. For stress testing beyond the declared ODD, we embed SLICESCORER within SLICENAV, an LLM-orchestrated verification pipeline where the modelinterprets developer queries to select relevant operators (triage, scoring, acquisition, evaluation) and vocabulary extensions, composing verification workflows whilekeeping all scoring deterministic and auditable. Experiments on three driving VLMs(WiseAD, DriveMM, Cosmos-Reason2-2B) show that SLICENAV surfaceshigh-risk coverage gaps more effectively than prior slice-discovery methods while maintaining diverse recommendations across the condition space. Ablations confirm both scoring components contribute, and qualitative analysis demonstrates end-to-end workflows from developer query to targeted evaluation.

Read more

Open-SAT: LLM-Guided Query Embedding Refinement for Open-Vocabulary Object Retrieval in Satellite Imagery

/in /by

In satellite applications, user queries often take the form of open-ended natural language, extending beyond a fixed set of predefined categories. This open-vocabulary nature poses significant challenges for retrieving relevant image tiles, as the retrieval system must generalize to a wide range of unseen objects and concepts. While vision-language models (VLMs) such as CLIP are widely used for text-image retrieval, even fine-tuned variants often struggle to accurately align such queries with satellite imagery. To address this, we propose Open-SAT, a training-free query embedding refinement algorithm that operates at inference time to improve alignment between user queries and satellite image content. Open-SAT uses VLMs to compute embeddings for image tiles, which are stored in a vector database for efficient retrieval. At query time, it leverages Large Language Models (LLMs) to refine the text embeddings by incorporating contextual information about objects of interest and their surroundings. A threshold-free retrieval mechanism further enhances accuracy and efficiency. Experimental results in three public benchmarks demonstrate that Open-SAT improves the F1 score by up to 16.04%, while retrieving a comparable number of image tiles. These results demonstrate the effectiveness of Open-SAT in open-vocabulary satellite image retrieval, leveraging LLM guidance without the need for additional training or supervision.

Read more

Closed-Form Statistical Modeling of PDL-Induced SNR Margins for Reliable Optical Networks

/in /by

We develop closed-form formulas for PDL-induced SNR margins using solutions based on central limit theorem. Experimental validations confirm accurate and conservative performancepredictions, enabling precise quality of transmission assessment and margin-aware design in optical networks.

Read more

RunAgent: Interpreting Natural-Language Plans with Constraint-Guided Execution (IEEE)

/in /by

Humans solve problems by executing targeted plans, yet large language models (LLMs) remain unreliable for structured workflow execution. We propose RunAgent, a multiagent plan execution platform that interprets natural-language plans while enforcing stepwise execution through constraints and rubrics. RunAgent bridges the expressiveness of natural language and the determinism of programming languages via an agentic language with explicit control constructs (e.g., IF, GOTO, FORALL). It autonomously derives and verifies constraints at each step; dynamically selects among LLM reasoning, tool use, and Python execution; and integrates error correction to ensure correctness. Evaluations on Natural-plan and SciBench Datasets demonstrate that RunAgent outperforms baseline LLMs and state-of-the-art PlanGEN methods.

Read more