MEDIA ANALYTICS
PROJECTS
PEOPLE
PUBLICATIONS
PATENTS
Media Analytics
Ziyu Jiang is a Senior Researcher in the Media Analytics Department at NEC Laboratories America in San Jose, CA. He earned his Ph.D. in Computer Science from Texas A&M University, where his doctoral work explored advanced methods in deep learning and visual understanding, laying the groundwork for his contributions to large-scale AI systems.
At NEC, Dr. Jiang’s work spans two core areas: autonomous driving and robotics. On the autonomous driving side, he focuses on building a simulator—a digital twin of the real world—to train and validate autonomous driving algorithms. On the robotics side, he is developing an agentic system that enables advanced robot deployment and self-evolution.
Drawing on his expertise across neural rendering, diffusion-based generation, 3D reconstruction, robotics, and agentic systems, Dr. Jiang’s innovations play a central role in NEC’s Autonomous Driving Simulation and Embodied AI initiatives. His work enables the automatic construction of photorealistic, controllable driving simulations, substantially reducing the cost of validating autonomous driving algorithms and ensuring their safety in long-tail scenarios. He is also taking on an increasingly important role in NEC’s robotics development.
Overview: Our AI DevOps pipeline builds a high-fidelity digital twin of sensor data which allows for self-improvement of deployed models. We leverage our foundational vision-language models to automatically determine issues in currently deployed AI, pseudo-label or simulate training data, develop models with continual learning and use an LLM-based verification over diverse scenarios.
Overview: Our simulation framework utilizes advances in neural rendering, diffusion models and large language models to automatically transform drive data into a full 3D sensor simulation testbed with unmatched photorealism.
Overview: Our face recognition methods achieve high accuracy on competitive public benchmarks through the use of universal representation learning techniques that leverage very large-scale datasets, with robustness to variations such as occlusions, blur, lighting or accessories.
Our HorizonWeaver software edits driving scenes with instruction-guided AI, adding traffic, changing weather, and generalizing to unseen roads, all while preserving the safety-critical details that keep autonomous vehicle testing honest.
Video generation models can look physically convincing while getting the physics completely wrong. PhyCo, new research from our Media Analytics department, introduces continuous, controllable physical properties to video AI, allowing practitioners to specify friction, bounce, and force.
Controllable driving scene generation is critical for realistic and scalable autonomous driving simulation, yet existing approaches struggle to jointly achieve photorealism and precise control. We introduce HorizonForge, a unified framework that reconstructs scenes as editable Gaussian Splats and Meshes,
Ensuring safety in autonomous driving requires scalable generation of realistic, controllable driving scenes beyond what real-world testing provides. Yet existing instruction guided image editors, trained on object-centric or artistic data, struggle with dense, safety-critical driving layouts. We propose
LangDriveCTRL is a natural-language-controllable framework for editing real-world driving videos to synthesize diverse traffic scenarios. It represents each video as an explicit 3D scene graph, decomposing the scene into a static background and dynamic object nodes. To enable fine-grained editing and
Modern video diffusion models excel at appearance synthesis but still struggle with physical consistency: objects drift, collisions lack realistic rebound, and material responses seldom match their underlying properties. We present PhyCo, a framework that introduces continuous, interpretable, and physically
NEC Labs America headed to Denver for CVPR 2026, one of the most prestigious gatherings in computer vision, machine learning, and pattern recognition. The IEEE/CVF Conference on Computer Vision and Pattern Recognition brought innovators from around the world to share breakthroughs.
Modern video diffusion models excel at appearance synthesis but still struggle with physical consistency: objects drift, collisions lack realistic rebound, and material responses seldom match their underlying properties. We present PhyCo, a framework that introduces continuous, interpretable, and physically
HorizonForge introduces a new approach to driving scene generation, enabling precise control over both vehicle behavior and identity. By allowing arbitrary trajectories and flexible vehicle insertion, it creates realistic, scalable simulations for autonomous driving, digital twins, and advanced AI development.
Ensuring safety in autonomous driving requires scalable generation of realistic, controllable driving scenes beyond what real-world testing provides. Yet existing instruction guided image editors, trained on object-centric or artistic data, struggle with dense, safety-critical driving layouts. We propose
Controllable driving scene generation is critical for realistic and scalable autonomous driving simulation, yet existing approaches struggle to jointly achieve photorealism and precise control. We introduce HorizonForge, a unified framework that reconstructs scenes as editable Gaussian Splats and Meshes,
This paper proposes AutoScape, a long-horizon driving scene generation framework. At its core is a novel RGB-D diffusion model that iteratively generates sparse, geometrically consistent keyframes, serving as reliable anchors for the scenes appearance and geometry. To maintain long-range geometric consistency,
The recent advent of large-scale 3D data, e.g. Objaverse, has led to impressive progress in training pose-conditioned diffusion models for novel view synthesis. However, due to the synthetic nature of such 3D data, their performance drops significantly when applied to real-world images. This paper consolidates
Photorealistic simulation plays a crucial role in applications such as autonomous driving, where advances in neural radiance fields (NeRFs) may allow better scalability through the automatic creation of digital 3D assets. However, reconstruction quality suffers on street scenes due to largely collinear
We address the challenging task of occlusion-aware indoor 3D scene understanding. We represent scenes by a set of planes, where each one is defined by its normal, offset and two masks outlining (i) the extent of the visible part and (ii) the full region that consists of both visible and occluded parts
