CLAP-S: Support Set Based Adaptation for Downstream Fiber-optic Acoustic Recognition

/in /by

Publication Date: 1/13/2025

Event: https://arxiv.org

Reference: https://arxiv.org/pdf/2501.09877

Authors: Jingchen Sun, NEC Laboratories America, Inc., University at Buffalo; Shaobo Han, NEC Laboratories America, Inc.; Wataru Kohno, NEC Laboratories America, Inc.; Changyou Chen, University at Buffalo

Abstract: Contrastive Language-Audio Pretraining (CLAP) models have demonstrated unprecedented performance in various acoustic signal recognition tasks. Fiber optic-based acoustic recognition is one of the most important downstream tasks and plays a significant role in environmental sensing. Adapting CLAP for fiber-optic acoustic recognition has become an active research area. As a non-conventional acoustic sensor, fiber-optic acoustic recognition presents a challenging, domain-specific, low-shot deployment environment with significant domain shifts due to unique frequency response and noise characteristics. To address these challenges, we propose a support-based adaptation method, CLAP-S, which linearly interpolates a CLAP Adapter with the Support Set, leveraging both implicit knowledge through fine-tuning and explicit knowledge retrieved from memory for cross-domain generalization. Experimental results show that our method delivers competitive performance on both laboratory-recorded fiber-optic ESC-50 datasets and a real-world fiber-optic gunshot-firework dataset. Our research also provides valuable insights for other downstream acoustic recognition tasks.

Publication Link: https://arxiv.org/pdf/2501.09877