Beyond echocardiography: toward clinical diagnosis through multimodal fusion
Date:
Echocardiography is the first-line modality for cardiac diagnosis, but building reliable automated tools on top of it is often hindered by data scarcity, interobserver variability, and domain shifts across centers and vendors. Recent foundation models illustrate a shift toward multimodal integration, combining ultrasound sequences with clinical reports for large-scale diagnosis and report generation, yet they still lack visual evidence clinicians can inspect, explicit uncertainty modeling, and interpretability. In this talk, I present an alternative strategy that leverages clinical knowledge to reduce reliance on massive datasets while reinforcing explainability. By extracting interpretable time series from ultrasound sequences — through robust segmentation, uncertainty estimation, and motion tracking — I show how these descriptors can be fused with patient tabular data to characterize arterial hypertension, using transformer-based tokenizers and an asymmetric fusion framework that improves both performance and interpretability.
