Header

Hero Section

Machine Learning

eigen Video Understanding

Aiming to achieve near-human video understanding, in this project we analyze several gigabytes of spatiotemporal data to perform action recognition, multi-person tracking, object permanence and video reasoning. eigen has built a scalable video-understanding platform for long-form video reasoning that scales to new environments and camera angles without any re-training.

eigen also provides a system platform. Running both on the cloud (AWS) and on-prem, it can scale up to thousands of streams into it for cloud-based AI processing. Our AI video algorithms provide efficient streaming and inference. eigen has a web frontend and support for iOS/Android playback. eigen has been tested in several retail POCs serving 200+ streams; its behavioral analytics have also been evaluated through various NEC customers. Using mixed precision and TensorRT, eigen is extremely efficient and incurs very low cloud costs.

Publications

Hopper: Multi-hop Transformer for Spatiotemporal Reasoning. In ICLR 2021. PDF. Poster. Honglu Zhou, Asim Kadav, Farley Lai, Alexandru Niculescu-Mizil, Martin Renqiang Min, Mubbasir Kapadia, Hans Peter Graf.

15 Keypoints is All You Need. Michael Snower, Asim Kadav, Farley Lai, Hans Peter Graf. In CVPR 2020. PDF. Ranked #1 in PoseTrack.

Tripping Through Time: Efficient Localization of Activities in Videos. (Spotlight) Meera Hahn, Asim Kadav, James M. Rehg, Hans Peter Graf. In CVPR Workship on Language and Vision, 2019. Also, appears in BMVC'20. PDF.

Visual Entailment: A Novel Task for Fine-Grained Image Understanding. Ning Xie, Farley Lai, Derek Doran, Asim Kadav. In NeurIPS Workshop on Visually-Grounded Interaction and Language, 2018 (ViGIL’18). PDF. Dataset. Check out the leaderboard!

Teaching Syntax using Adverserial Distraction. Juho Kim, Christopher Malon, and Asim Kadav. In FEVER-EMNLP, 2018. PDF.

Attend and Interact: Higher-Order Object Interactions for Video Understanding. Chih-Yao Ma, Asim Kadav, Iain Melvin, Zsolt Kira, Ghassan AlRegib, and Hans Peter Graf. In CVPR, 2018. PDF. Video. Live Demo.

Collaborators: 

Hans Peter Graf

Farley Lai

Asim Kadav

Eigen Collaborators: Asim Kadav (Lead), Farley Lai, Deep Patel, Rowena Chen, Anupriya Prasad, Shayan Bhatti, Shrey Shah, Ramakrishnan Sundareswaran, Likitha Lakshminarayan, Aagam Shah, Maxwell Springer, Aaron Rosati

sinet