DFA-RAG: Conversational Semantic Router for Large Language Model with Definite Finite Automaton

Publication Date: 7/27/2024

Event: The Forty-first International Conference on Machine Learning (ICML 2024), Vienna, Austria

Reference: pp. 1-23, 2024

Authors: Yiyou Sun, NEC Laboratories America, Inc.; Junjie Hu, University of Wisconsin; Wei Cheng, NEC Laboratories America, Inc.; Haifeng Chen, NEC Laboratories America, Inc.

Abstract: This paper introduces the retrieval-augmented large language model with Definite Finite Automaton (DFA-RAG), a novel framework designed to enhance the capabilities of conversational agents using large language models (LLMs). Traditional LLMs face challenges in generating regulated and compliant responses in special scenarios with predetermined response guidelines, like emotional support and customer service. Our framework addresses these challenges by embedding a Definite Finite Automaton (DFA), learned from training dialogues, within the LLM. This structured approach acts as a semantic router which enables the LLM to adhere to a deterministic response pathway. The routing is achieved by the retrieval-augmentation generation (RAG) strategy, which carefully selects dialogue examples aligned with the current conversational context. The advantages of DFA-RAG include an interpretable structure through human-readable DFA, context-aware retrieval for responses in conversations, and plug-and-play compatibility with existing LLMs. Extensive benchmarks validate DFA-RAG’s effectiveness, indicating its potential as a valuable contribution to the conversational agent.

Publication Link: https://openreview.net/forum?id=LpAzlcGzJ6

Additional Publication Link: https://arxiv.org/pdf/2402.04411