Cognita Imaging has secured a $1.29 million research contract from the U.S. Food and Drug Administration (FDA) to develop and validate a framework for evaluating AI-generated radiology reports using multiple large language models (LLMs).
The 18-month contract, which took effect June 22, centers on what Cognita calls an "LLMs-as-a-jury" approach, which compares the judgments of several LLMs rather than relying on a single model to grade a report. After building and validating the framework, the team will apply it to approximately one million patient exams from a diverse U.S. cohort, examining performance across patient groups, care settings, imaging equipment, and diseases.
Radiologists will review clinically significant disagreements to distinguish errors in AI-generated reports from those in the LLM evaluation itself, with findings used to refine the framework and inform both premarket validation and post-market monitoring approaches.
Cognita is a wholly owned subsidiary of Mosaic Clinical Technologies, which is in turn a wholly owned subsidiary of Radiology Partners.
