Building the Brain of Dr. Algorithm

Archimedes advances evidence-based medicine to foster model-based medicine

This posting is by guest author Tuan Dinh, who will speak about this topic at the Strata Rx conference.

Legendary Silicon Valley investor Vinod Khosla caused quite a stir last year when he predicted at Strata Rx that “Dr. Algorithm”–artificial intelligence driven by large data sets and computational power–would replace doctors in the not-too-distant future. At that point, he said, technology will be cheaper, more accurate and objective, and will ultimately do a better job than the average human doctor at delivering routine diagnoses with standard treatments.

I not only support Khosla’s provocative prophecy, I’ll add one of my own: that Dr. Algorithm (aka Dr. A) will “come to life” in three to five years, by the time today’s first-year med school students are pulling 30-hour shifts as new interns. But what will it take to build the brain of Dr. A? And how can we teach Dr. A to account for increasingly complex medical inputs, such as laboratory tests results, genomic/genetic information, family and personal history, co-morbidities and patient preferences, so he can make optimal clinical decisions for living, breathing patients?

The current paradigm for clinical decision making is Evidence-Based Medicine (EBM), in the forms of evidence-based practice and evidence-based guidelines. Can EBM be the brain of Dr. A? The problem is that while EBM is the status-quo of how clinical decisions are made today, it’s far from ideal given its many limitations. First, randomized clinical trials (RCTs), the gold standard evidence of EBM, do not address all questions and scenarios. RCTs are designed to get FDA approval hence they do not always employ protocols and enroll patients that area representative of the real world.

Second, systematic reviews, the labor-intensive process of evidence retrieval, grading and meta-analysis, simply can’t handle the data explosion that’s come with electronic medical records (EMRs), proteomics. and genomics.

Third, evidence-based guidelines, while ubiquitous today, were designed for pre-computer area so that physicians can easily memorize them. The guidelines today tend to focus on one variable at a time (e.g., blood pressures), and using sharp thresholds (e.g., systolic blood pressure > 140 mm Hg) for treatment recommendations, ignoring the continuous nature of physiological variables and the complexity of disease progression.

Model-based medicine (MBM)–the use of large-scale integrated physiology and pathology-driven mathematical models to translate and synthesize existing evidence and medical knowledge into a unified framework–has recently emerged as a basis to address these challenges. In my view, MBM is EBM 2.0–an upgrade, if you will–in the ability to make relevant clinical decisions at both individual and population level. It not only incorporates all available evidence and the most up-to-date understanding of diseases, but also accounts for uncertainties in data and gaps in knowledge.

MBM serves as a crossing point between evidence and physicians, allowing the rapid extraction of quantitative, robust and already synthesized information for customized clinical decision-making. The decisions can be optimized not only based on the therapeutic efficacy of health interventions and current health status of patients, but also on a patient’s preferences and health behavior (such as their prior likelihood to comply with treatment recommendations).

We already see the role of modeling in clinical decision-making today. For individual patients, regression-based models are increasingly integrated into the work flow to stratify risks and assess benefits of treatments. Several examples, well-regarded among the medical community, easily come to mind:

  • The Framingham risk score, based on data from the famed heart study, is a gender-specific algorithm to estimate an individual’s 10-year cardiovascular risk.
  • The Gail risk score, an interactive tool developed by scientists at the National Cancer Institute, among others, estimates a woman’s risk of developing invasive breast cancer by taking seven key risk factors into account.
  • The Apgar score, which has been in place for more than 60 years, quickly evaluates the health of newborns just after birth using five factors.
  • At a population level, a noteworthy example of MBM shaping clinical decisions includes cancer screening guidelines (for breast and lung cancer) issued by the U.S. Preventive Services Task Force, which are based on Cancer Intervention and Surveillance Modeling Network (CISNET) models.

But we can’t assume that these established examples mean that the path toward a more thorough adoption of MBM will be a straight and smooth one. In fact, many challenges remain. With data availability, we have a Wild West situation, where almost anybody with data will develop a model. For instance, at least 10 established models exist for predicting cardiovascular diseases, while 50 are in place for diabetes onset and 100 models estimate prostate cancer risk.

But just because a model exists doesn’t mean it’s accurate, and hence useful. And this is the seminal obstacle to proceeding with MBM: there are currently no standards or regulations guiding how to develop, validate, calibrate and evaluate model. There is a growing concern that the majority of models are poorly developed and plagued by technical issues such as over-fitting to biased patient sample, lack of transferability to another population or setting, questionable handling/selection of risk predictors, and questionable treatment of missing data.

Adding to the problem, many physicians remain unconvinced about the credibility and usability of clinical models. Many models employ structures or make assumptions that lack face validity and are inconsistent with conventional medical knowledge.

Complicating matters is the fact that few decision makers–i.e., doctors–have the training needed to understand, and to accept models. Physicians are quick to point to the fact that different models provide different answers to the same question. Some models are just too opaque to non data scientists. Physicians frequently complain that they can not understand how an artificial neural network or support vector machine algorithm might reach a particular decision.

To advance MBM, I believe that we need to do four things.

  1. On the technical side, we need to solve the problem of how to integrating evidence across data sources (e.g. clinical trials, EMRs, disease registries, observational studies, expert opinions), scale (e.g. clinical, genomics, physiological), and diseases.
  2. We need to develop standards and regulations to quantify accuracy and uncertainties in model predictions.Organizations such as the International Society of Pharmacoeconomics Outcomes Research (ISPOR) have already made headway in this direction.
  3. We need to develop tools for decision makers (i.e. physicians, patients, policy makers) to easily interact with models.
  4. Finally, we need to educate physicians about evaluating models and being comfortable using model predictions in their everyday work.

At Archimedes, we’re working to resolve the scientific and technical obstacles to MBM. Our Archimedes Model, a large-scale simulation system of pathophysiology, interventions, and healthcare systems, was built from thousands of data sources and rigorously validated against hundreds more. We also built two software platforms that enable non-modelers to interface with the Archimedes Model: ARCHeS for population-level decisions and IndiGO for individual patient decisions.

Ultimately, as daunting as the challenges in adopting model-based medicine may seem, I maintain that we’re not far off from building Dr. A. The key to this success is a joint effort by data scientists and physicians to build common standards and software platforms. If we work together, Dr. A will be a reality in three-to-five years.

Tuan Dinh, PhD, is vice president of analytics and modeling for Archimedes, Inc., a leading healthcare analytics company based in San Francisco. He has 15 years of experience developing advanced analytics and modeling solutions to understand and optimize complex systems, including nuclear power plants, biological cells, human physiology and pathology and healthcare systems. His recent work spans several therapeutic areas, including prevention, screening and treatment of cancers, diabetes and cardiovascular diseases, genomic and genetic testing, mental health and medication adherence.

tags: , , , , , ,