Deep Learning for Computer-Assisted Interventions in Cataract Surgery

By Dr Silpaja Chandrasekar, PhDApr 23 2024Reviewed by Susha Cheriyedath, M.Sc.

In a paper published in the journal Scientific Data, researchers highlighted the transformative impact of deep-learning (DL) techniques on computer-assisted interventions and post-operative surgical video analysis. The emphasis was on the crucial role of large-scale datasets and annotations in advancing DL-powered surgical technologies in areas like surgical scene understanding and phase recognition.

The largest cataract surgery video dataset was tailored to support computerized surgical workflow analysis and post-operative irregularity detection. Annotation quality was validated through benchmarking against state-of-the-art neural network architectures. Pioneering research on domain adaptation for instrument segmentation in cataract surgery followed, with the subsequent release of the dataset and annotations on Synapse.

Background

Past work has seen the emergence of context-aware systems (CAS) in evolving operation rooms to facilitate pre-operative planning, skill assessment, operation room planning, and comprehensive surgical context interpretation. These systems offer real-time alerts and decision-making support for experienced and less-experienced surgeons. Cataract surgery presents an ideal domain for DL applications due to its complexity and global impact on visual impairment.

Technological advancements have helped in the evolution of cataract surgery techniques and increased interest in DL methodologies for analyzing surgical videos. However, existing public datasets for cataract surgery are limited in scope, hindering the development of comprehensive deep-learning-based approaches. There is a pressing need for large-scale datasets with multi-task annotations to advance cataract surgery outcomes.

Experimental Methodologies Overview

The experimental phase recognition methodology combined convolutional neural networks (CNNs) with a recurrent neural networks (RNNs) framework. The CNN component extracted features from individual frames, and the RNN component captured temporal features from video sequences. Visual geometry group 16-layer (VGG16) and residual network (ResNet50) architectures were employed using different pre-trained CNN architectures.

Four RNN architectures (long short-term memory (LSTM), gated recurrent unit (GRU), bidirectional LSTM (BiLSTM), and BiGRU) were compared for performance. The training settings involved segmenting videos into three-second clips, augmenting data with various techniques, and employing a random sampling strategy for diversity. The team applied dropout regularization during training. The models were trained using a binary cross-entropy loss function and Adam optimizer, with performance evaluated based on accuracy and F1 score.

Experiments were conducted to validate the effectiveness of pixel-level annotations using various state-of-the-art baselines targeting general images, medical images, and surgical videos for semantic segmentation. The analysts initialized backbones with pre-trained parameters from ImageNet. The training involved utilizing a combination of cropping, rotation, color jittering, blurring, and sharpening augmentations. The cross-entropy log dice loss function was employed during training, with parameters set to prevent overfitting and optimize performance. Performance evaluation was based on average dice and average intersection over union (IoU) metrics. Phase recognition and semantic segmentation experiments followed a systematic approach involving network architecture selection, data preprocessing, augmentation strategies, loss function definition, and performance evaluation metrics.

Multi-task Annotation Validation

Researchers validating the quality of multi-task annotations involved rigorously training several state-of-the-art neural network architectures for each task. The performance of the trained models is evaluated using relevant metrics to ensure the accuracy and reliability of annotations. 

The investigation into phase recognition performance unveils commendable results across a spectrum of CNN-RNN architectures. Bidirectional recurrent layers were incorporated to reveal consistent performance enhancements by combining certain phases with shared visual features. Additionally, networks utilizing the ResNet50 backbone exhibited marginally superior performance to those employing VGG16, owing to the deeper architecture's efficacy in extracting intricate features essential for accurate recognition.

A comprehensive quantitative analysis is provided for segmenting relevant anatomical structures and instruments. Notably, pupil segmentation demonstrated the highest performance due to its distinct features, while instrument segmentation encountered significant challenges such as motion blur and reflections. Interestingly, the DeepPyramid network with a VGG16 backbone consistently yielded optimal results across all classes.

A visual comparison of average dice and IoU metrics across five folds for evaluated neural networks is presented. Here, networks like DeepPyramid, adaptive network (AdaptNet), and residual calibration network (ReCal-Net) emerged as the top performers for anatomy and instrument segmentation in cataract surgery videos to showcase the promising potential for accurate segmentation tasks.

Lastly, the team shed light on the performance disparity between intra-domain and cross-domain scenarios using binary instrument annotations. Results indicate notable differences between the Cataract-1k and CaDIS datasets, highlighting the substantial domain shift inherent in these datasets. This underscores the importance of exploring semi-supervised and domain adaptation techniques to enhance instrument segmentation performance in cross-dataset domain shifts.

Conclusion

In summary, the meticulous validation process of multi-task annotations ensures their precision and dependability, achieved through extensive training in cutting-edge neural network architectures. The evaluation of phase recognition performance reveals promising outcomes across various CNN-RNN configurations, with bidirectional recurrent layers consistently enhancing accuracy.

Additionally, the deeper architecture of backbone networks, like ResNet50, contributes to slightly superior performance compared to VGG16. The comprehensive quantitative analysis conducted for segmenting relevant anatomical structures and instruments sheds light on the efficacy of different segmentation tasks.

While pupil segmentation emerges as highly successful, challenges such as motion blur and reflections hinder instrument segmentation. Notably, the DeepPyramid network with a VGG16 backbone consistently demonstrates optimal results across all classes. It underscores the significance of advanced neural network architectures in achieving accurate segmentation in complex medical imaging scenarios.