Decoding refers to the process used by large language models to generate text output from a given input. This process involves selecting words or tokens from a probability distribution generated by the model to form coherent and contextually appropriate sentences. Different decoding methods determine how these selections are made to achieve various goals, such as maximizing likelihood, maintaining diversity, or avoiding factual inaccuracies.

 

Posts

Self-Consistent Decoding for More Factual Open Responses

/in /by

Self-consistency has emerged as a powerful method for improving the accuracy of short answers generated by large language models. As previously defined, it only concerns the accuracy of a final answer parsed from generated text. In this work, we extend the idea to open response generation, by integrating voting into the decoding method. Each output sentence is selected from among multiple samples, conditioning on the previous selections, based on a simple token overlap score. We compare this “Sample & Select” method to greedy decoding, beam search, nucleus sampling, and the recently introduced hallucination avoiding decoders of DoLa, P-CRR, and S-CRR. We show that Sample & Select improves factuality by a 30% relative margin against these decoders in NLI-based evaluation on the subsets of CNN/DM and XSum used in the FRANK benchmark, while maintaining comparable ROUGE-1 F1 scores against reference summaries. We collect human verifications of the generated summaries, confirming the factual superiority of our method.

Read more