AI ultrasound model accurately estimates gestational age

An AI model based on blind sweep ultrasound data collected by novice operators can accurately estimate gestational age and be generalizable, according to research published July 9 in JAMA Network Open. 

The model AI model achieved noninferiority to clinical standard estimates in two healthcare settings in the U.S. and Africa, wrote a team led by Angelica Willis from Google Research in Mountain View, CA, and colleagues. 

“Our findings offer critical insights into the challenges and opportunities for deploying AI in clinical healthcare settings,” the Willis team wrote. 

AI systems that use low-cost portable ultrasound devices address operator dependency while also bridging gaps in maternal care. One approach is blind sweep ultrasonography, a set of protocolized sweeps that does not rely on real-time imaging interpretation. 

While AI ultrasound systems have shown success in recent studies, the researchers noted that these systems must be generalizable to serve various patient populations with unique circumstances. 

Willis and colleagues investigate the performance of a previously trained AI system, a neural network model trained on the Fetal Age Machine Learning Initiative (FAMLI) cohort. They tested the system’s ability to interpret blind sweep ultrasonography scans for gestational age estimation and generalize to new clinical environments. 

Datasets for the study included the following: an adaptation set of 120 participants in Chicago, Il, split evenly for training and validation); and a primary evaluation set of 385 participants (192 in Chicago and 193 in Nairobi, Kenya) with gestational ages of 16 to 36 weeks. The team also used a one-day noninferiority margin when comparing the respective performances of the AI system and standard clinical assessment. 

The adapted AI model achieved a mean average error (MAE) of 4.1 days in Chicago and 4.3 days in Nairobi. Overall, the model achieved an MAE of 4.2 days compared to 4.5 days for standard clinical assessment, achieving noninferiority (p < 0.001 for noninferiority). 

The Chicago cohort had two operators while the Nairobu cohort had three operators, all having no prior ultrasound experience. The Chicago-based operators relied on a protocol handout, verbal instructions, and a brief demonstration. This was revised for the Nairobi-based operators, who received six hours of hands-on training. The model had a lower sweep rejection rate in Nairobi compared to Chicago (1.8% vs. 7.9%). 

The study authors highlighted the AI model’s potential to expand access to diagnostic tasks in low-resource settings by enabling novice operators to perform accurate ultrasonography assessments. 

Despite the model’s success, the study authors pointed out infrastructural hurdles that act as barriers toward adopting this AI approach. These include “unreliable power for charging devices and the need for durable hardware capable of withstanding demanding conditions.” 

“Furthermore, these advanced AI models require building trust with clinicians and patients, via transparency, such as implementing model confidence metrics to prevent predictions on low-quality data, and by ensuring the model’s fairness and equity through ethically sourced, diverse training data and validation across multiple sites,” the authors wrote. 

Read the full study here.