CLAP-S: Support Set Based Adaptation for Downstream Fiber-optic Acoustic Recognition

/in /by

Publication Date: 4/9/2025

Event: 2025 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2025)

Reference: pp. 1-5, 2025

Authors: Jingchen Sun, NEC Laboratories America, Inc., University at Buffalo; Shaobo Han, NEC Laboratories America, Inc.; Wataru Kohno, NEC Laboratories America, Inc.; Changyou Chen, University at Buffalo

Abstract: Contrastive Language-Audio Pretraining (CLAP) models have demonstrated unprecedented performance in various acoustic signal recognition tasks. Fiber-optic-based acoustic recognition is one of the most important downstream tasks and plays a significant role in environmental sensing. Adapting CLAP for fiber-optic acoustic recognition has become an active research area. As a non-conventional acoustic sensor, fiberoptic acoustic recognition presents a challenging, domain-specific, low-shot deployment environment with significant domain shifts due to unique frequency response and noise characteristics. To address these challenges, we propose a support-based adaptation method, CLAP-S, which linearly interpolates a CLAP Adapter with the Support Set, leveraging both implicit knowledge through fine-tuning and explicit knowledge retrieved from memory for cross-domain generalization. Experimental results show that our method delivers competitive performance on both laboratory recorded fiber-optic ESC-50 datasets and a real-world fiber optic gunshot-firework dataset. Our research also provides valuable insights for other downstream acoustic recognition tasks.

Publication Link: https://ieeexplore.ieee.org/document/10887829