Beta-KD (Beta-weighted Knowledge Distillation) is an uncertainty-aware framework for transferring knowledge from a large teacher model to a compact student model. Formulated from a Bayesian perspective, it interprets teacher supervision as a Gibbs prior over student activations, producing a closed-form weighting mechanism that dynamically balances learning from training data and teacher guidance. Beta-KD is designed for multimodal large language models and addresses instability caused by capacity gaps between teacher and student architectures.
Machine learning is shifting from learning from data alone to learning from both data and teacher models. Beta-KD uses uncertainty-aware Bayesian weighting to train compact multimodal AI without blindly trusting every teacher signal.
Princeton Office
4 Independence Way, Suite 200
Princeton, NJ 08540
+1 609-520-1555
San Jose Office
2033 Gateway Place, Suite 200
San Jose, CA 95110
+1 408-863-6007
NEC Laboratories America, Inc. (NEC Labs) is the US-based center for NEC Corporation’s global network of corporate research laboratories. Our diverse research groups collaborate with industry, academia and governments to provide disruptive solutions to complex problems. A leader in the integration of IT and network technologies with more than 100 years of expertise, NEC provides a combination of products and solutions that cross-utilize the company’s experience and global resources to meet the complex and ever-changing needs of its customers.