Faithful Medical Dialogue Generation Using Homo-Heterogeneous Exemplar-based In-Context Knowledge Grounding

Abstract

The growing reliance on tele-healthcare has heightened the demand for accessible and professional health support. Artificial Intelligence (AI)-assisted medical dialogue systems have emerged as key solutions, with Large Language Models (LLMs) advancing the generation tasks. However, their susceptibility to hallucination leads to inaccurate and unreliable information, posing major challenges. To address this, we propose a novel approach to mitigate hallucinations in LLMs by integrating external knowledge and in-context learning mechanisms for faithful medical dialogue generation (MDG). In particular, we devise an In-context Medical Knowledge-grounded Dialogue Generator (IMKDG), a novel plug-and-play retrieval-based framework that leverages external medical knowledge, in-context learning (ICL), and retrieval methods to enable LLMs to generate faithful responses, thereby enhancing their performance on the MDG task. We utilize large-scale medical knowledge based on the Unified Medical Language System (UMLS) to retrieve knowledge pertinent to the dialogue context. Further, to enhance the LLMs’ ICL capability for the MDG task, we propose the Homo-Heterogeneous Exemplar Selection (H2ES) method, a novel in-context exemplar retrieval method based on both dialogue context and medical knowledge. Automatic and human evaluations on the MedDialog-EN and CDialog datasets across various LLMs demonstrate the efficacy of the proposed framework in mitigating hallucinations.

Resources

Details