LLMs rapidly evolving in nuclear medicine

Large language models are reshaping patient data management and clinical practice in nuclear medicine, a study has reported.

In a review of published literature, researchers found that since 2023, the number of tasks in the field to which LLMs have been applied jumped from 41 to 434, with a majority of models being explored for use in information processing.

“LLMs represent a transformative evolution from traditional [natural language processing] approaches in nuclear medicine, offering unprecedented capabilities that extend far beyond the rule-based and statistical methods previously employed in the field,” noted lead author Pia Koller, a doctoral student at the University of Bern in Switzerland, and colleagues. The study was published on October 21 in PET Clinics.

While general medicine and other fields in radiology have seen a wave of reviews exploring the capabilities of LLMs, there remains a notable gap in the literature focusing specifically on LLMs in nuclear medicine and theranostics, the authors noted.

To address the gap, the group systematically analyzed current and emerging applications of LLMs in the field.

The researchers limited their search of the literature to peer-reviewed articles in English published from 2023 onward, reflecting the Chatbot hype after the release of ChatGPT in November 2022. A targeted analysis returned 174 publications: 101 tagged with precision medicine, 59 with nuclear medicine, and 10 with theranostics. Four publications belonged to multiple areas.

The most frequently studied models were from major AI providers, particularly the OpenAI generative pretraining transformer (GPT) family (ChatGPT, GPT-3.5, and ChatGPT-4) and the Google Gemini family. Beyond these dominant players, models from the BERT family, Meta LLaMA family, and Anthropic Claude family are also being investigated, as well as vision-language models such as GPT-4o, GPT-4V, Claude-3 Sonnet, and Claude-3 Opus.

The researchers categorized the applications for the models by how they fit into a clinician’s workflow: information processing and data management, clinical decision support, diagnostic and imaging, and medical education and training.

“LLMs are widely used to handle the large volume of text data generated in nuclear medicine, where accurate interpretation and processing of complex imaging and clinical reports are critical,” the researchers wrote.

They noted that the models are mostly being used to assist in tasks such as summarizing lengthy patient histories or scientific literature, extracting key findings from nuclear medicine reports, and structuring unstructured clinical notes into organized, accessible formats.

Ultimately, before the release of ChatGPT in November 2022, there were multiple attempts to use natural language processing (NLP) in nuclear medicine focused on structured data extraction from reports and basic text classification tasks using rule-based and statistical methods, the researchers wrote.

However, LLMs come with critical challenges over NLP systems, including reliability concerns (hallucinations), the need for specialized domain knowledge, multimodal data integration complexities, standardized evaluation frameworks, and heightened ethical and privacy considerations, they added.

“By addressing these challenges through targeted research and clinical validation, LLMs can fulfill their potential as transformative tools in specialized medical practice,” the group concluded.

The full study is available here.