Controllable Dynamic Multi Task Architectures

/in /by

Publication Date: 3/23/2022

Event: arXiv

Reference: https://arxiv.org/abs/2203.14949

Authors: Dripta Raychaudhuri, University of California, Riverside, NEC Laboratories America, Inc., Yumin Suh, NEC Laboratories America, Inc., Samuel Schulter, NEC Laboratories America, Inc., Xiang Yu, NEC Laboratories America, Inc., Masoud Faraki, NEC Laboratories America, Inc., Amit K. Roy Chowdhury, University of California, Riverside, Manmohan Chankdraker, NEC Laboratories America, Inc., University of California, San Diego

Abstract: Multi task learning commonly encounters competition for resources among tasks, specifically when model capacity is limited. This challenge motivates models which allow control over the relative importance of tasks and total compute cost during inference time. In this work, we propose such a controllable multi task network that dynamically adjusts its architecture and weights to match the desired task preference as well as the resource constraints. In contrast to the existing dynamic multi task approaches that adjust only the weights within a fixed architecture, our approach affords the flexibility to dynamically control the total computational cost and match the user preferred task importance better. We propose a disentangled training of two hypernetworks, by exploiting task affinity and a novel branching regularized loss, to take input preferences and accordingly predict tree structured models with adapted weights. Experiments on three multi task benchmarks, namely PASCAL Context, NYU v2, and CIFAR 100, show the efficacy of our approach. Project page is available at https://www.nec labs.com/˜mas/DYMU.

Publication Link: https://arxiv.org/pdf/2203.14949.pdf

You might also like
5 Accepted Papers CVPR2022! Check out our Computervision Papers at #CVPR2022
5 Accepted Papers CVPR2022! Active Adversarial Domain Adaptation
5 Accepted Papers CVPR2022! Adaptation Across Extreme Variations using Unlabeled Bridges
5 Accepted Papers CVPR2022! Adversarial Alignment for Source Free Object Detection
5 Accepted Papers CVPR2022! Adversarial Learning of Privacy-Preserving and Task-Oriented Representations
5 Accepted Papers CVPR2022! Channel Recurrent Attention Networks for Video Pedestrian Retrieval
5 Accepted Papers CVPR2022! Controllable Dynamic Multi-Task Architectures
5 Accepted Papers CVPR2022! Coordinated Joint Multimodal Embeddings for Generalized Audio-Visual Zero-shot Classification and Retrieval of Videos