A HALF-DAY TUTORIAL OFFERED AT MICCAI 2023, VANCOUVER

Surviving the Valley of Death:

DESIGNING MACHINE LEARNING SOLUTIONS TO REACH THE CLINIC

Developing machine learning (ML) solutions for eventual clinical deployment requires a different approach than experimental work done in the context of advancing state-of-the-art (SOTA) models. Global Health Labs, Inc. (GH Labs) is honored to host a MICCAI tutorial focused on key elements for researchers to consider during the initial research phase so that their ML solutions can survive the "valley of death" between the research lab and clinic.

SCHEDULE
DATE
TIME
LOCATION
REGISTRATION

October 12, 2023

Afternoon PDT (half day) - full schedule below

MICCAI, Vancouver

Please visit the MICCAI Conference website to register and obtain streaming instructions for virtual attendance

Photo © B. Mellor, Nature 2008.

Abstract

Developing ML solutions for eventual clinical deployment requires a different approach than experimental work done in the context of advancing state-of-the-art (SOTA) models. Most published ML methods perish in the "valley of death" between the research lab and clinic, often because crucial clinical issues and constraints are underappreciated and thus ignored during initial research. By actively addressing a few key themes during the initial research phase, ML researchers can set up their models for successful deployment.

This tutorial will discuss how these key themes can be handled to prepare for successful deployment to the clinic. This includes design choices for exploratory ML projects, so that even if a project does not seek or reach clinical viability its findings can effectively guide future work, and thus truly contribute to a clinical deployment that benefits patients.

Concrete examples will be drawn from image-based ML projects in global health, including malaria diagnosis, cervical cancer prevention, and lung ultrasound. The tutorial will also provide a window on global health challenges and how ML can actively mitigate global health care inequalities.  

The tutorial themes include:

  • Understanding the deployment ecosystem
  • Centering the medical use-case and performance requirements
  • Tailoring metrics and loss functions to the use-case
  • Data -collecting, annotating, and cleaning
  • Emphasizing the central role of domain experts and field partners  
  • Highlighting productization concerns

Participants will gain insights about key themes central to any ML project aiming for clinical translation and will acquire concrete tools to optimize for deploying in clinics. Participants will also gain insights into the global health landscape and how ML can benefit currently underserved populations.

Tutorial Outline

This half-day tutorial will include several talks, and ample time for Q & A and networking.

SCHEDULE
1:30-1:40 PDT
1:40-2:30 PDT
2:30-2:40 PDT
2:40-3:30 PDT

Welcome and Introduction of Themes - Daniel Shea

Keynote: Creating and Translating AI Innovations to Solutions for Poor and Underserved Communities - P. Anandan

RISE (recruiting LMIC researchers) - Andrea Lara

Key Insights on Clinical Implementation from a 25-year Career in Radiology AI - Ron Summers

3:30-4:00 PDT

Break

4:00-4:30 PDT

Key Challenges in Medical Data and Annotation - Courosh Mehanian

4:30-5:00 PDT

Designing Lung Ultrasound Models for Clinical Integration - Daniel Shea

5:00-5:30 PDT

Case Study: An AI Solution for Detecting Low Birthweight Newborns - P. Anandan

5:30-5:40 PDT

Closing Remarks - Courosh Mehanian

UNDERSTANDING THE DEPLOYMENT
ECOSYSTEM

To move beyond the sandbox, a proposed ML solution must meet the concrete needs of patients, practitioners, and the business landscape.

Photo © Denis Ngai. Pexels 2020.

CENTERING THE MEDICAL USE-CASE AND
PERFORMANCE REQUIREMENTS

A proposed method that violates field realities and constraints (e.g., available gear or currently in-use protocols), or which does not directly address clinical performance needs, is almost certain to fail however fine it may look in theory.

THE CENTRAL ROLE OF DOMAIN EXPERTS
AND FIELD PARTNERS

It is crucial to seek out advice from experts who know the details of the medical use-case and care context. Domain experts are central to all the themes listed above.

LUNG ULTRASOUND: CLINICAL GROUND TRUTH CHALLENGES

A case study highlighting pitfalls and challenges at all research phases, when developing ML for deployment.

Photo © Bill & Melinda Gates Foundation/Frederic Courbet

PREPARATION FOR DEPLOYMENT AS A PRODUCT

Some regulatory requirements need to be designed into the early stages of algorithm development, to prevent dead ends. For example, FDA requirements may limit the choice of ML methods.

SPEAKERS
P. Anandan
Microsoft Research India, Wadhwani Institute for AI

A renowned researcher in computer vision and artificial intelligence, Anandan’s career spans over 30 years in academia and industry in the US and in India. He established Microsoft Research India in 2005, after expanding the computer vision program at Microsoft Research at Microsoft’s US headquarters. He has also served as the CEO of the Wadhwani Institute for AI, which focuses on developing AI for social good.

Ronald Summers
Director of Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, National Institutes of Health

Dr. Ronald M. Summers is a pioneer in the use of artificial intelligence in radiology. His lab has made seminal contributions to the advancement of cancer diagnosis using radiology. He has received numerous awards, including the Presidential Early Career Award for Scientists and Engineers, the NIH Director’s Award, and the NIH Clinical Center Director’s Award.

Courosh Mehanian  
Principal Research Engineer, Machine Learning Group, GH Labs

Courosh Mehanian received a physics PhD from Cornell University. He has worked for three decades in artificial intelligence, having held positions at Boston University, MIT Lincoln Laboratory, the University of Oregon and now GH Labs, where his lab applies ML and computer vision to automated medical image understanding.

Andrea Lara
Director of BiomedLab, Galileo University

Andrea Lara is the director and founder of BiomedLab, Galileo University, Guatemala's first biomedical engineering research laboratory. Her research field is medical image processing and analysis, focused mainly on cardiac CT. She has a Ph.D. from the Graz University of Technology, a master's degree in biomedical engineering from the University of Lübeck, and worked as a research assistant at the Fraunhofer Institute for Biomedical Engineering.

Daniel Shea
Research Engineer, Machine Learning Group, GH Labs

Daniel Shea is a research machine learning engineer at GH Labs where his work focuses on computer vision artificial intelligence for radiological and health applications. His previous work includes machine learning for nonlinear materials, and biomedical laboratory research on organ-on-a-chip systems, nucleic acid rapid diagnostics, nano- and micro-fabrication, and mesoporous functional biomaterials.

About Global Health Labs

Global Health Labs, Inc. (GH Labs) innovates to reduce health disparities, especially in low- and middle-income countries. As a nonprofit corporation fully funded by Gates Ventures (the private office of Bill Gates), we partner with the Bill & Melinda Gates Foundation and other cross-sector leaders to develop technology solutions that address unmet needs in:

  • Diagnostics
  • Maternal, Newborn, and Child Health.
  • Primary Healthcare Tools and Equipment.

Recent innovations from GH Labs resulted in several commercially available rapid COVID-19 antigen tests, an AI-enabled smartphone app to detect cervical cancer, and a portable cold chain device that enables vaccine delivery to remote communities—among many other tools and technologies advanced for the people who need them most.

Contact Us

charles.delahunt@ghlabs.org

(+ 1) 425.777 .9687

Global Health Labs
14360 SE Eastgate Way
Bellevue, WA, 98007-6462

Organizing Committee

  • Charles Delahunt
    Senior Research Engineer, Machine Learning Group, GH Labs, Bellevue, WA
  • Noni Gachuhi
    Program Lead for Maternal, Newborn, and Children's Health Portfolio, GH Labs, Bellevue, WA
  • Matt Horning
    Director, Machine Learning Group, GH Labs, Bellevue, WA
  • Courosh Mehanian
    Principal Research Engineer, Machine Learning Group, GH Labs, Bellevue, WA; Affiliate Faculty, Bioengineering, University of Oregon, Eugene, OR
  • Daniel Shea
    Research Engineer, Machine Learning Group, GH Labs, Bellevue, WA

Resources

Coming soon!