Unsupervised and Semi-Supervised Domain Adaptation for Action Recognition from Drones

Publication Date: 3/2/2020

Event: WACV 2020, Snowmass Village, CO USA

Reference: pp 1706-1715, 2020

Authors: Jinwoo Choi, Virginia Tech, NEC Laboratories America, Inc.; Gaurav Sharma, NEC Laboratories America, Inc.; Manmohan Chandraker, NEC Laboratories America, Inc.; Jia-Bin Huang, Virginia Tech

Abstract: We address the problem of human action classification in drone videos. Due to the high cost of capturing and labeling large-scale drone videos with diverse actions, we present unsupervised and semi-supervised domain adaptation approaches that leverage both the existing fully annotated action recognition datasets and unannotated (or only a few annotated) videos from drones. To study the emerging problem of drone-based action recognition, we create a new dataset, NEC-DRONE, containing 5,250 videos to evaluate the task. We tackle both problem settings with 1) same and 2) different action label sets for the source (e.g., Kinectics dataset) and target domains (drone videos). We present a combination of video and instance-based adaptation methods, paired with either a classifier or an embedding-based framework to transfer the knowledge from source to target. Our results show that the proposed adaptation approach substantially improves the performance on these challenging and practical tasks. We further demonstrate the applicability of our method for learning cross-view action recognition on the Charades-Ego dataset. We provide qualitative analysis to understand the behaviors of our approaches.

Publication Link: https://ieeexplore.ieee.org/document/9093511