?

EndoPlan-3D is developing an AI-assisted, interactive 3D environment to improve the planning of endoscopic procedures for gastrointestinal early neoplasia. By fusing multimodal data from white-light endoscopy and endoscopic ultrasound using AI algorithms, individualized, dynamic 3D models are generated. These models are designed to support physicians in a VR environment by enabling accurate identification of tumor boundaries, critical hotspots, and optimal resection strategies. The 新万博体育下载_万博体育app【投注官网】 of Augsburg leads the project’s participatory design process. Through interviews and co-design workshops with endoscopy experts, a user-centered 3D environment is being developed to support realistic and effective surgical planning. Speech data generated during the “think-aloud” planning process is analyzed using large language models to enable automated, speech-based annotation. In addition, the NovA annotation tool, developed at the 新万博体育下载_万博体育app【投注官网】 of Augsburg, is used to systematically capture surgical planning workflows - including time-stamped critical events relevant to both planning and execution. The 新万博体育下载_万博体育app【投注官网】 of Augsburg is also responsible for investigating the ethical, legal, and social aspects (ELSA) of clinical integration and for developing corresponding guidelines.