Advances in surgical techniques are being assisted by a new approach to evaluate the performance of tissue-tracking software.
Surgeons need a detailed view of tissues when performing complex, non-invasive procedures using robot assistance, making advances in tissue imaging computer software essential to minimise errors. The Surgical Tattoos in Infrared (STIR) technology designed by Vancouver Coastal Health Research Institute researcher Dr. Tim Salcudean and Dr. Adam Schmidt provides a novel way to evaluate tissue tracking algorithms, setting the stage for future advances in image-guided and automated surgeries.
“Tissues have repetitive textures that can prove problematic for computer algorithms used in robot-assisted surgeries to accurately track,” notes Schmidt.
Schmidt and Salcudean’s research, carried out in collaboration with Intuitive Surgical, the makers of da Vinci robotic surgery systems, was published in IEEE Transactions on Medical Imaging. The study releases a dataset to enable any researcher to evaluate how well their tissue tracking computer algorithms are able to track the movements of various tissue types viewed through a da Vinci Xi robot. For their study, Salcudean and Schmidt tested STIR on three baseline algorithms.
The da Vinci Xi is a robotic surgical system used in endoscopy — a procedure that involves the insertion of a small video camera-mounted rigid endoscope stick and instruments to view and operate inside of the body in a minimally invasive manner.
Surgeons using the da Vinci Xi sit in front of a console with ‘stereo video’ goggles showing real-time three-dimensional video of interior tissues. They manipulate the machine’s ‘arms’ to control the movement of the video camera-mounted scope and surgical tools.
Tissue tracking and mapping software augments the surgical process further by superimposing images on top of the stereo video display to, for example, highlight target tissues, such as tumours.
“Computer software gives surgeons a view beyond that of the endoscopic camera, such as vasculature and other areas that surgeons want to avoid, as well as highlighting the tissues to be removed,” says Salcudean.
“Augmented reality is being used to improve precision surgical techniques for enhanced patient outcomes and recovery.”
However, software errors or blind spots can lead to the surgical removal of healthy tissues, with potential downsides for patient outcomes and recovery times.
Optimizing precision surgical techniques with tissue tracking software
The STIR approach designed by Schmidt and Salcudean involved tattooing onto tissues non-toxic, near-infrared dyes invisible to the human eye, but viewable through the da Vinci Xi system’s monitor, or similar specialised equipment. The system captures both near-infrared and visible light spectrum video, which made it possible for the research team to toggle between both in a single recording.
The tissue types examined were pork chop, aorta, beef tongue, chicken heart, pig heart, chicken breast, pork stomach, pork intestine and pork spleen, notes Schmidt. Two in vivo porcine studies were also conducted. “To enable a similar lighting environment to that found in a surgical setting, we placed tissues into a body model that replicates the lighting found when performing surgery in the torso.”
Near-infrared dyes were tattooed in multiple places on each tissue sample. Researchers then recorded tissue manipulation and movement, toggling between the near-infrared and visible light views. Simulated tissue manipulations ranged from total movement to stretching and squishing, as well as camera movements, palpation, cutting, tearing and both instruments and other tissues obscuring the view of the camera.
“Collecting a diverse set of actions helped us evaluate the performance of algorithms under various conditions.”
Because the computer algorithms tested were only able to track tissues from the visible light footage, researchers could compare their performance against the footage with the invisible light annotated dye markings to determine the algorithms’ accuracy.
While the team proved STIR to be a reliable approach to evaluate the performance of tissue-tracking algorithms, they also discovered that most algorithms had flaws that made their ability to track tissues inadequate for the clinical setting, says Schmidt.
“Accuracy levels dropped significantly when the image was obscured by a tool or other tissues, for example. Large tissue movements also resulted in reductions in accuracy.”
“STIR can be used to improve algorithms designed for robot-guided surgery, optimizing them before they are further validated in clinical settings.”
In the long-term Salcudean and Schmidt envision applications for STIR to include evaluating software used for robot-guided prostate and other surgeries.
“Augmented surgical applications, such as surgical robots, are the way forward for precision surgical techniques and optimal patient outcomes,” says Salcudean. “By further enhancing these approaches through proven, optimized augmented reality software, we can get to the point where the surgeon is presented with extensive information to make the best decision for patient outcomes.”
