Resource Data

Proof-of-concept model uses AI to combine multiple types of data to predict cancer outcomes

Although it has long been recognized that predicting outcomes in cancer patients requires consideration of many factors, such as patient history, genes, and disease pathology, clinicians struggle to integrate this information to make decisions about patient care. A new study by researchers from the Mahmood Lab at Brigham and Women’s Hospital reveals a proof-of-concept model that uses artificial intelligence (AI) to combine multiple types of data from different sources to predict patient outcomes for 14 different types of cancer. The results are published in Cancer cell.

Experts depend on multiple sources of data, such as genomic sequencing, pathology, and patient history, to diagnose and prognosticate different types of cancer. Although existing technology allows them to use this information to predict outcomes, manually integrating data from different sources is difficult and experts often find themselves making subjective assessments.

Experts analyze lots of evidence to predict how well a patient may be doing. These early reviews become the basis for decision-making regarding enrollment in a clinical trial or specific treatment regimens. But that means that this multimodal prediction happens at the expert level. We are trying to solve the problem by calculation.”

Faisal Mahmood, PhD, assistant professor in the division of computational pathology at Brigham and associate member of the cancer program at the Broad Institute of Harvard and MIT

Using these new AI models, Mahmood and his colleagues discovered a way to integrate multiple forms of computationally diagnostic information to achieve more accurate outcome predictions. AI models demonstrate the ability to make prognostic determinations while uncovering the predictive bases of characteristics used to predict patient risk; a property that could be used to discover new biomarkers.

The researchers built the models using The Cancer Genome Atlas (TCGA), a publicly available resource containing data on many types of cancer. They then developed a deep learning-based multimodal algorithm capable of learning prognostic information from multiple data sources. By first creating separate models for histological and genomic data, they could merge the technology into a single integrated entity that would provide key prognostic information. Finally, they evaluated the efficiency of the model by feeding it datasets from 14 cancer types as well as patient histology and genomic data. The results demonstrated that the models produced more accurate predictions of patient outcomes than those that incorporated only one source of information.

This study highlights that it is possible to use AI to integrate different types of clinically informed data to predict disease outcomes. Mahmood explained that these models could allow researchers to discover biomarkers that integrate different clinical factors and better understand the type of information they need to diagnose different types of cancer. Researchers also quantitatively investigated the importance of each diagnostic modality for individual cancer types and the benefit of integrating multiple modalities.

AI models are also able to elucidate pathological and genomic features that drive prognostic predictions. The team found that the models used patients’ immune responses as a prognostic marker without being trained to do so, a notable finding given that previous research shows that patients whose tumors elicit stronger immune responses tend to get better results.

Although this proof-of-concept model reveals a new role for AI technology in cancer care, this research is only a first step in the clinical implementation of these models. The application of these models in the clinic requires the incorporation of larger data sets and validation on large cohorts of independent tests. In the future, Mahmood aims to incorporate even more types of patient information, such as X-ray exams, family history and electronic medical records, and possibly take the model to clinical trials.

“This work paves the way for larger studies of AI in healthcare that combine data from multiple sources,” Mahmood said. “In a broader sense, our findings underscore the need to build computational pathology prognosis models with much larger datasets and downstream clinical trials to establish utility.”


Brigham and Women’s Hospital

Journal reference:

Chen, RJ, et al. (2022) Pan-cancer integrative histological-genomic analysis via multimodal deep learning. Cancer cell.