Predictive care, i.e. data driven prediction and risk stratification on an individual level, enables prevention, early intervention and tailored therapies. Predictive care is considered a cornerstone for future healthcare. Multimodal large language models (MLLM) with their few-shot learning capabilities and ability to perform various tasks have high potential even for prediction tasks but remain black boxes.

?

The project COGITAMUS-Health focuses on basic considerations to use MLLM for predictions and use their ability for self-reasoning. We aim to create a systematic review on methods to elicit reasoning from MLLM on multimodal medical data and implement as well as evaluate a method for deterioration prediction from publicly available multimodal data (e.g. biosignals, electronical health records, text notes from MIMIC-IV) as an exemplary use-case for our MLLM-based prediction and reasoning.

?

?

The project is a collaboration between the Chair of Diagnostic Sensing at the 新万博体育下载_万博体育app【投注官网】 of Augsburg and the Laboratory of Diagnostic and Interventional Adaptive Imaging (IADI) at INSERM in France.