eigen projects aims to achieve near-human video understanding and video reasoning. In this project, we analyze several gigabytes of spatio-temporal data to perform action recognition, multi-person tracking, object permanence and video reasoning. eigen has built a scalable video understanding platform for long form video reasoning that scales to new environments and camera angles without any re-training.

eigen also provides a system platform. eigen runs on the cloud (AWS) and on-prem and can scale upto thousands of streams into it for cloud-based AI processing. Our AI video algorithms provide efficient streaming and inference. eigen has a web frontend and support for iOS/Android playback. eigen has been tested in several retail POCs serving 200+ streams as well as various NEC customers for behavioral analytics through its business units. Finally, eigen is extremely efficient using mixed precision and TensorRT and incurs very low cloud costs.

NEC Labs: Asim Kadav, Farley Lai, Hans Peter Graf

EIGEN Team: Asim Kadav (Lead), Farley Lai, Deep Patel, Rowena Chen, Anupriya Prasad, Shayan Bhatti, Shrey Shah, Ramakrishnan Sundareswaran, Likitha Lakshminarayan, Aagam Shah, Maxwell Springer, Aaron Rosati

eigen website: EIGEN



  • Hopper: Multi-hop Transformer for Spatiotemporal Reasoning. In ICLR 2021. PDF. Poster. Honglu Zhou, Asim Kadav, Farley Lai, Alexandru Niculescu-Mizil, Martin Renqiang Min, Mubbasir Kapadia, Hans Peter Graf.
  • Tripping Through Time: Efficient Localization of Activities in Videos. (Spotlight) Meera Hahn, Asim Kadav, James M. Rehg, Hans Peter Graf.
    In CVPR Workship on Language and Vision, 2019. Also, appears in BMVC'20. PDF.
  • Visual Entailment: A Novel Task for Fine-Grained Image Understanding. Ning Xie, Farley Lai, Derek Doran, Asim Kadav.
    In NeurIPS Workshop on Visually-Grounded Interaction and Language, 2018 (ViGIL’18). PDF.Dataset. Check out the leaderboard!
  • Teaching Syntax using Adverserial Distraction. Juho Kim, Christopher Malon, and Asim Kadav. In FEVER-EMNLP, 2018. PDF.
  • Attend and Interact: Higher-Order Object Interactions for Video Understanding. Chih-Yao Ma, Asim Kadav, Iain Melvin, Zsolt Kira, Ghassan AlRegib, and Hans Peter Graf. In CVPR, 2018. PDF. Video. Live Demo.