Artificial intelligence (AI) is all around us and is becoming an increasingly important tool for healthcare.
“We have a great opportunity to bring together medicine and artificial intelligence,” says Gil Shamai, of the Department of Computer Science at Technion-Israel Institute of Technology, who is currently working on his postdoc in the Institute’s Geometric Image Processing Laboratory led by Professor Ron Kimmel. His Ph.D. work centered around how AI enhances and can revolutionize cancer research.
It all started after a conversation with a physician and clinician for oncology. “He told me artificial intelligence, deep learning in particular, is capable of doing amazing things,” Shamai says. “And he suggested this idea of implementing it for cancer research, and that became a big part of my Ph.D.”
The work began with images of cancer biopsies, to which Shamai applied deep learning models. He has been working with Kimmel and an expanding team of computer scientists and clinicians to develop expertise in deep learning models for analysis of cancer biopsy images.
Applying deep learning to breast cancer biopsy analysis
They looked at breast cancer biopsies, in particular, because incidences of this form of cancer continue to increase as people enjoy longer lifespans. The World Health Organization reports in its most recent statistics available that 2.3 million women were diagnosed with breast cancer globally in 2020, making it the most prevalent form of cancer.
Shamai and his team use deep learning models to analyze the biopsy images to find correlations between them and the expression of different receptors, which are proteins on the membrane of the cells.
“The knowledge of the existence of these receptors or, in other words, the expression of these receptors, is crucial in cancer care. It’s crucial to profile the cancer in them,” Shamai says, noting that in every case of breast cancer, diagnosis must be done via the receptors in estrogen, progesterone, and HER2.
“It's like sub-diluting the cancer in a molecular way,” he continues. “It gives you better knowledge of how to treat it, and to determine the prognosis and other things that are important in a sub-type of the cancer.”
Currently, seeing such receptors that exist on cells is impossible with a conventional microscope and without a special staining. These cancer receptors are not viewable with this technique, so an advanced staining approach called immunohistochemistry is necessary. This laboratory method uses antibodies usually linked to a fluorescent dye or enzyme to check for certain markers (receptors) in a cell or tissue sample.
While such staining-centric techniques have been used for the past several decades, Shamai notes they present challenges. “It’s not always accurate and the staining doesn’t always work, and you can’t always know that right away; it takes time,” he says. “And it all depends on the type of cancer and the receptors.”
Cost is also a factor, as is availability and feasibility, particularly in remote and economically challenged regions.
“So we found a different way,” Shamai says. “Our idea was to see if AI/deep learning computer models can predict if there is expression of specific receptors.”
His team first inspected estrogen receptors to determine positive or negative expression status, and whether the computer model could infer this by simply looking at the images (see Fig. 1). “We managed to prove, for the first time, that artificial intelligence can extract information from biopsy images about the molecular profile of the cancer,” Shamai says.
This could all tie into immunotherapy, which is treatment that uses a person’s own immune system to fight cancer by attacking tumors and cancerous cells. This type of therapy is personalized so each patient receives the proper medications and care based on specific characteristics of the type of cancer, tumors, and cell receptors. AI/deep learning models will be key to advancing immunotherapy.
Along with receptors, Shamai’s team is demonstrating the deep learning model on a specific protein that’s displayed in some tumors. “It's a protein, an agent, called PD-L1, which is crucial for understanding if a cancer patient will benefit from immunotherapy,” he says.
PD-L1 essentially keeps the body’s immune responses under control. This protein can be found on some normal cells and in higher concentrations on some types of cancer cells and tumors. One concern is that PD-L1 can mistakenly convince the immune system to leave the cancer alone. Immunotherapy has the ability to overrule that and actually persuade the immune system to attack the cancer that expresses PD-L1.
“With this treatment, you attempt to make the immune system of the body attack the cancer,” Shamai says. “This protein is an indicator if treatment is going to work, and you need to know if it exists or not. We showed that you can do this just by this deep learning analysis of the biopsy images without using the advanced immunohistochemistry staining.”
Predicting PD-L1 expression of tumors
In their recent study, published in Nature Communications, Shamai, Kimmel, and Amir Livne, a graduate student and fellow researcher at Technion, worked with researchers from the Institute of Molecular Pathology and Immunology at the University of Porto (Portugal) and Carmel Medical Center (Haifa, Israel) to develop an artificial neural network to predict the PD-L1 expression of the tumor.
The network reviewed breast cancer images from biopsies obtained from Vancouver General Hospital (British Columbia, Canada) to determine which cells expressed the PD-L1 protein and which did not (see Fig. 2). It was able to successfully apply the method to 70% of the patients.
Some cases were also reviewed by a human pathologist, who performed the diagnosis and quantified the expression using the advanced immunohistochemistry staining, while the AI predicted the expression without it—in cases where that person and the AI network disagreed, subsequent tests proved the AI findings were correct, because it is able to distinguish certain characteristics that even a highly trained and experienced pathologist simply cannot see.
Shamai calls this “a significant achievement. It's something that is going to be an integral part of cancer research, treatments, and diagnosis.”
The research team of Shamai and Kimmel has expanded over the past several years and now includes several Master’s and Ph.D. students developing the computational models, as well as a dozen undergraduate students working on difference projects. A couple of clinicians are also helping with data collection and quality assurance. Today, the team is collaborating with pathologists and oncologists from four medical centers in Israel, including Sheba, Carmel, Ichilov, and Haemek.
They are now expanding its studies and AI techniques to other types of cancer and to predict other related aspects such as a patient’s survival time, based on the analysis of the shape of the cells and the arrangement of the cells in the tissue. Knowing the survival time could allow doctors to better determine best treatment options.
“The information is in images of the biopsies and it's beyond what pathologists can see. We know that because the pathologist does not have the capabilities of the AI. They do not know even how the AI is doing it. Even though we read the algorithm and we know that computation, we don't understand it, so a human being cannot repeat it,” Shamai says. “Basically, what we show is that there is information that was not exploited until now.”
