MEDS: Building Models and Tools in a Reproducible Health AI Ecosystem

Abstract

Health AI suffers from a systemic reproducibility crisis that irreparably hinders research in this space across academia and industry. To combat this and empower researchers in the health AI space, we propose a comprehensive interactive tutorial introducing the ''Medical Event Data Standard'' (MEDS) and its growing open-source ecosystem. Working in MEDS allows you to more easily build AI models over public or private longitudinal EHR datasets and to readily benchmark existing, published models against contributions on local datasets and tasks. MEDS simplifies the construction of AI models on longitudinal Electronic Health Record (EHR) datasets and enables straightforward benchmarking against established models. Reflecting its growing adoption, MEDS is utilized at over 15 institutions across 8 countries, features 7+ open-source tools, supports 10+ published models, and provides publicly available Extract-Transform-Load (ETL) pipelines for major public EHR datasets. A KDD tutorial offering practical experience with MEDS will significantly enhance reproducibility and comparability in health AI research. In this tutorial, we will teach attendees how to (1) transform datasets into the MEDS format(2) pre-process MEDS data for modeling needs(3) build highly effective, efficient, AI models for diverse predictive tasks on their datasets, and (4) contribute their results to MEDS-DEV, a decentralized benchmark enabling robust evaluation against meaningful baselines. Participants will engage in collaborative, minimal-dependency Jupyter notebook exercises, guided through each step by structured instruction and practical coding sessions. Attendees will leave equipped with practical knowledge to build reproducible, state-of-the-art AI models within the MEDS ecosystem.