Yang Li is a former research intern in the Optical Networking & Sensing department of NEC Laboratories America, Inc. while studying at Georgia State University.

Posts

VB-LoRA: Extreme Parameter Efficient Fine-Tuning with Vector Banks

As the adoption of large language models increases and the need for per-user or per-task model customization grows, the parameter-efficient fine-tuning (PEFT) methods, such as low-rank adaptation (LoRA) and its variants, incur substantial storage and transmission costs. To further reduce stored parameters, we introduce a “divide-and-share” paradigm that breaks the barriers of low-rank decomposition across matrix dimensions, modules, and layers by sharing parameters globally via a vector bank. As an instantiation of the paradigm to LoRA, our proposed VB-LoRA composites all the low-rank matrices of LoRA from a shared vector bank with a differentiable top-k admixture module. VB-LoRA achieves extreme parameter efficiency while maintaining comparable or better performance compared to state-of-the-art PEFT methods. Extensive experiments demonstrate the effectiveness of VB-LoRA on natural language understanding, natural language generation, instruction tuning, and mathematical reasoning tasks. When fine-tuning the Llama2-13B model, VB-LoRA only uses 0.4% of LoRA’s stored parameters, yet achieves superior results. Our source code is available at https://github.com/leo-yangli/VB-LoRA. This method has been merged into the Hugging Face PEFT package.

Improvement of Resilience of Submarine Networks Based on Fiber Sensing

Simultaneous phase and polarization sensing with span length resolution using the supervisory path is demonstrated. It is shown that by measuring polarization rotation matrix of the return paths, instead of monitoring only the state of polarization, location of the polarization disturbance can be determined even for large polarization rotations. By using the polarization rotation matrices, the phase and polarization disturbances are successfully decoupled. How the existing supervisory system and sensing can coexist in new SDM cables that utilizes pump sharing is discussed.

Polarization Sensing Using Polarization Rotation Matrix Eigenvalue Method

Polarization-based, multi-span sensing over a link with reflection-back circuits is demonstrated experimentally. By measuring rotation matrices instead of just monitoring polarization, a 35 dB extinction in localization is achieved regardless of the disturbance magnitude.