MEDIA ANALYTICS
PROJECTS
PEOPLE
PUBLICATIONS
PATENTS
Our simulation framework utilizes advances in neural rendering, diffusion models and large language models to automatically transform drive data into a full 3D sensor simulation testbed with unmatched photorealism. We offer language-based control to generate safety-critical scenarios such as collisions, traffic rule violations and other unsafe behaviors, to improve the perception and planning abilities of autonomous vehicles. While traditional simulation frameworks require expertise from artists, graphics engineers and game engine developers, ours is designed to empower end users with simulation abilities that leverage their domain knowledge. Our simulation frameworks allow the development of autonomous vehicle AI stacks without the need for exhaustive road testing, while ensuring that training data encompasses diverse real-world conditions.
Team Member: Yumin Suh, Bingbing Zhuang, Ziyu Jiang
Autonomous vehicle (AV) systems rely on robust perception models as a cornerstone of safety assurance. However, objects encountered on the road exhibit a long-tailed distribution, with rare or unseen categories posing challenges to a deployed perception model. This necessitates an expensive process of
Photorealistic simulation plays a crucial role in applications such as autonomous driving, where advances in neural radiance fields (NeRFs) may allow better scalability through the automatic creation of digital 3D assets. However, reconstruction quality suffers on street scenes due to largely collinear
Monocular 3D object localization in driving scenes is a crucial task, but challenging due to its ill-posed nature. Estimating 3D coordinates for each pixel on the object surface holds great potential as it provides dense 2D-3D geometric constraints for the underlying PnP problem. However, high-quality
Simulation is a useful tool in situations where training data for machine learning models is costly to annotate or even hard to acquire. In this work, we propose a reinforcement learning-based method for automatically adjusting the parameters of any (non-differentiable) simulator, thereby controlling
