Uncertainty-Aware Knowledge Distillation for Multimodal Large Language Models

Publication Date: 6/3/2026

Event: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026)

Reference: pp. 5585-5595, 2026

Authors: Jingchen Sun, NEC Laboratories America, Inc., University at Buffalo, SUNY; Shaobo Han, NEC Laboratories America, Inc.; Deep Patel, NEC Laboratories America, Inc.; Wataru Kohno, NEC Laboratories America, Inc.; Can Jin, Rutgers University; Changyou Chen, Rutgers University

Abstract: Knowledge distillation establishes a learning paradigm that leverages both data supervision and teacher guidance. However, determining the optimal balance between learning from data and learning from the teacher is challenging, as some samples may be noisy while others are subject to teacher uncertainty. This motivates the need for adaptively balancing data and teacher supervision. We propose Beta-weighted Knowledge Distillation (Beta-KD), an uncertainty-aware distillation framework that adaptively modulates how much the student relies on teacher guidance. Specifically, we formulate teacher-student learning from a unified Bayesian perspective and interpret teacher supervision as a Gibbs prior over student activations. This yields a closed-form, uncertainty-aware weighting mechanism and supports arbitrary distillation objectives and their combinations. Extensive experiments are conducted on multimodal VQA benchmarks by distilling a student Vision-Language Model from a large teacher VLM. The results demonstrate that Beta-KD consistently outperforms existing knowledge distillation methods. Code is available at https://github.com/Jingchensun/beta-kd.

Publication Link: https://openaccess.thecvf.com/content/CVPR2026/html/Sun_Uncertainty-Aware_Knowledge_Distillation_for_Multimodal_Large_Language_Models_CVPR_2026_paper.html