Recent work by German Cancer Research Center (DKFZ) researchers is introducing artificial intelligence (AI) into healthcare settings by combining AI with multispectral imaging. Their system, which uses a multispectral camera and AI models, is the first such device to achieve video-rate recording and analysis (>20 Hz; see video).
“Our community currently lacks video-rate spectral imaging devices that can be used in clinical practice,” says Leonardo Ayala, research lead and a Ph.D. student in the division of Intelligent Medical Systems at DKFZ. The resulting system offers numerous advantages including production of higher-quality data and less image blurring, which occurs with organ motion when conventional techniques are used.
“Our development pioneers an entirely new imaging paradigm based on spectral imaging,” he says.
The work began with questioning the possibility of better monitoring perfusion—the passage of fluid through the lymphatic system and blood vessels to an organ or a tissue—during minimally invasive kidney surgery without having to inject a contrast agent into a patient’s bloodstream.
To date, they have used their AI-multispectral imaging system to automatically monitor perfusion and also ischemia (a condition causing a lack of blood flow to organs in the body) during kidney tumor surgery. The system enables them to view the optical properties of the tissue without contrast agents.
Surgeons are increasingly turning to minimally invasive laparoscopic procedures—commonly called keyhole surgeries—to deal with such tumors, which must be removed surgically. This involves a laparoscope equipped with a camera being inserted through a small incision in the abdomen to allow doctors to view affected organs and more efficiently remove tumors.
The laparoscope camera delivers image signals from the red, green, and blue wavelengths, but certain optical properties in human tissue are not represented in that information. DKFZ’s multispectral camera technology captures not only those three standard wavelengths, but also multispectral measurements over 16 additional wavelength ranges. This allows doctors to see in real-time a tissue’s functional properties, such as perfusion, invisible to conventional cameras and without fluorescent contrast agents.
AI models aid the multispectral camera with the ability to distinguish perfused from non-perfused tissue in individual patients, resulting in personalized treatment. “The new AI-multispectral imaging system has several implications both scientifically and for society,” Ayala says. “The development of a new technology that allows safe, real-time functional imaging during surgery implies substantial benefits for society through an improvement of both general surgical safety as well as specific patient outcomes.”
Removing the need for the injection of a contrast agent minimizes associated risks including anaphylactic shock. At the same time, the ability to easily reassess perfusion on demand during surgery minimizes the risk of surgical complications that might arise from malperfusion, which improves patient outcomes.
“We’ve had some exciting first results in a surgical procedure called partial nephrectomy, where part of the kidney affected by a tumor is removed,” Ayala says. “But we realized the kidney tissue from different patients looked very different, which meant developing a method that works in a group of patients might mean it will completely fail in a different group of patients.”
The team overcame this problem with an approach that “is highly personalized in that it exclusively leverages data derived from the individual patient in question,” Ayala says. “This safeguards the analysis of each patient’s data from potential confounders known to arise from training AI models on a patient pool that may not resemble the patient being treated.”
Another bonus: the system corresponds with hardware already existing in clinics, which minimizes overhead expenses.
There are several potential future steps, including the use of different light sources and camera chips, mainly to leverage the potential of other wavelength regions. For example, light in the near-infrared wavelength range generally reaches deeper regions of tissue, which limits undesirable surface effects such as scars or burned tissue.
“Our ultimate goal is to develop safe imaging techniques that can be translated into clinical practice,” Ayala says. “We expect this development to kick off a new direction in surgical imaging research with further improvements for both patients and clinical staff.”
