Multimodal contrastive learning

Medical diagnosis often involves several different kinds of information. This work explores how to learn useful representations across MRI images and tabular clinical features, using multimodal contrastive learning and tabular attention.

The ICCV 2023 work achieved Alzheimer’s prediction accuracy above 83.8%. A subsequent adaptive graph construction framework extends contrastive learning to an arbitrary number of modalities, improving generalizability. That work appeared in the NeurIPS 2024 High School Projects track, placing in the top four of 335 projects.

My contribution

• Built a multimodal framework combining MRI with tabular clinical features.
• Used tabular attention to improve Alzheimer’s disease prediction.
• Developed adaptive graph construction to learn from varying numbers of modalities.