Deep Learning Architectures and Techniques refer to the specific neural network structures and models used in deep learning, which is a subfield of machine learning. These architectures often involve multiple layers of artificial neurons (deep neural networks) and use techniques such as convolutional neural networks (CNNs) for image analysis, recurrent neural networks (RNNs) for sequential data processing, and transformers for natural language processing. Deep learning techniques encompass a wide range of methods for training and optimizing these architectures, including backpropagation, stochastic gradient descent (SGD), and various activation functions like ReLU (Rectified Linear Unit).

Posts

Controllable Dynamic Multi-Task Architectures

/in /by

Multi-task learning commonly encounters competition for resources among tasks, specifically when model capacity is limited. This challenge motivates models which allow control over the relative importance of tasks and total compute cost during inference time. In this work, we propose such a controllable multi-task network that dynamically adjusts its architecture and weights to match the desired task preference as well as the resource constraints. In contrast to the existing dynamic multi-task approaches that adjust only the weights within a fixed architecture, our approach affords the flexibility to dynamically control the total computational cost and match the user-preferred task importance better. We propose a disentangled training of two hyper networks, by exploiting task affinity and a novel branching regularized loss, to take input preferences and accordingly predict tree-structured models with adapted weights. Experiments on three multi-task benchmarks, namely PASCAL-Context, NYU-v2, and CIFAR-100, show the efficacy of our approach. Project page is available at https://www.nec-labs.com/-mas/DYMU.

Read more