The Point of View of a Sentiment: Towards Clinician Bias Detection in Psychiatric Notes

Abstract

Negative patient descriptions and stigmatizing language can contribute to generating healthcare disparities in two ways: (1) read by patients, they can harm their trust and engagement with the medical center; (2) read by physicians, they may negatively influence their perspective of a future patient. In psychiatry, the patient-clinician therapeutic alliance is a major determinant of clinical outcomes. Therefore, language usage in psychiatric clinical notes may not only create healthcare disparities, but also perpetuate them. Recent advances in natural language processing systems have facilitated the efforts to detect discriminatory language in healthcare. However, such attempts have only focused on the perspectives of the medical center and its physicians. Considering both physicians’ and non-physicians’ subjective points of view is a more equitable approach to identifying harmful language in clinical notes. By leveraging large language models (LLMs), this work aims to characterize potentially harmful language usage in psychiatric notes by identifying the sentiment expressed in sentences describing patients based on the reader’s point of view. First, we curated a psychiatric lexicon containing words commonly used to describe patients in psychiatry. Sentences (N=39) were extracted from clinical text containing psychiatric lexicon at a medical center, with which a set of physicians (N=10) and non-physicians (N=10) annotated them as negative, neutral, or positive. Three LLMs (GPT-3.5, Llama-3.1, and Mistral) used zero-shot/few-shot in-context learning (ICL) approaches to classify the sentiment of the sentences according to the physician or non-physician point of view. Results showed that GPT-3.5 aligned best to physician point of view and Mistral aligned best to non-physician point of view, both with an ICL approach. These results underline the importance of recognizing subjectivity in clinical annotation tasks, not only for improving the note writing process, but also for the quantification, identification, and reduction of bias in computational systems for downstream analyses.

Resources

Details