Skip to content

MJAHMADEE/Awesome_Video-guided_Surgery

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 

Repository files navigation

Awesome-Surgical-Video-Analysis 🎥🔬

Awesome Under Development

🚀 Under development. Welcome to contribute! 🌟

📝 TO DO:

  • 🖇 Add links
  • 🔍 Search for more papers
  • 📂 Better categories, Better curation

📚 Contents


🌟 The Big Picture

Title Venue Links
Surgical Data Science: from Concepts to Clinical Translation Arxiv 2020 Paper
Surgical Data Science: A Consensus Perspective Arxiv 2018 Paper
Surgical data science for next-generation interventions NBE 2017 Paper
CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer-Assisted Interventions IEEE 2019 Paper
OR Black Box and Surgical Control Tower: Recording and Streaming data and Analytics to Improve Surgical Care JVS 2021 Paper

📖 Survey

Title Venue Links
Machine learning for technical skill assessment in surgery: a systematic review NPJ 2022 Paper
A Review on Deep Learning in Minimally Invasive Surgery IEEE Access 2021 Paper
Deep neural networks for the assessment of surgical skills: A systematic review IEEE Access 2021 Paper
Current methods for assessing technical skill in cataract surgery JCRS 2021 Paper
Gesture Recognition in Robotic Surgery: a Review TBE 2021 Paper
Objective assessment of surgical technical skill and competency in the operating room ARBE 2017 Paper
Vision-based and marker-less surgical tool detection and tracking: a review of the literature MIA 2017 Paper
Surgical process modelling: a review IJCARS 2013 Paper
Image Processing and Machine Vision in Surgery and Its Training JoC 2023 Paper

🎯 Surgical Skill Assessment

Title Venue Links
Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmark Arxiv 2021 Paper
Video-based Surgical Skill Assessment using Tree-based Gaussian Process Classifier Arxiv 2023 Paper
Development and Validation of a 3-Dimensional Convolutional Neural Network for Automatic Surgical Skill Assessment Based on Spatiotemporal Video Analysis JAMA 2021 Paper
Surgical Skill Evaluation From Robot-Assisted Surgery Recordings ISMR 2021 Paper
Towards Unified Surgical Skill Assessment CVPR 2021 Paper Code
Clearness of Operating Field: A Surrogate for Surgical Skills on In-Vivo Clinical Data IJCARS 2020 Paper
Construction of Quantitative Indexes for Cataract Surgery Evaluation Based on Deep Learning OMIA 2020 Paper
Predicting the quality of surgical exposure using spatial and procedural features from laparoscopic videos IJCARS 2020 Paper
The Pros and Cons: Rank-aware Temporal Attention for Skill Determination in Long Videos CVPR 2019 Paper Code
Surgical Skill Assessment on In-Vivo Clinical Data via the Clearness of Operating Field MICCAI 2019 Paper
Modeling Surgical Technical Skill Using Expert Assessment for Automated Computer Rating AnnS 2019 Paper
Towards Optimizing Convolutional Neural Networks for Robotic Surgery Skill Evaluation IJCNN 2019 Paper
Video-based surgical skill assessment using 3D convolutional neural networks IJCARS 2019 Paper Code
Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks IJCARS 2019 Paper Code
Machine learning methods for automated technical skills assessment with instructional feedback in ultrasound-guided interventions IJCARS 2019 Paper
Objective assessment of intraoperative technical skill in capsulorhexis using videos of cataract surgery IJCARS 2019 Paper
Objective classification of psychomotor laparoscopic skills of surgeons based on three different approaches IJCARS 2019
Who’s Better? Who’s Best? Pairwise Deep Ranking for Skill Determination CVPR 2018
Tool Detection and Operative Skill Assessment in Surgical Videos Using Region-Based Convolutional Neural Networks WACV 2018
Evaluating surgical skills from kinematic data using convolutional neural networks MICCAI 2018 Paper Code
Automated surgical skill assessment in RMIS training IJCARS 2018
Video and accelerometer-based motion analysis for automated surgical skills assessment IJCARS 2018
Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery IJCARS 2018
Surgical skills: Can learning curves be computed from recordings of surgical activities? IJCARS 2018 Paper
Automated robot‐assisted surgical skill evaluation: Predictive analytics approach IJMRCAR 2018
Meaningful Assessment of Surgical Expertise: Semantic Labeling with Data and Crowds MICCAI 2016
Automated video-based assessment of surgical skills for training and evaluation in medical schools IJCARS 2016
Task-Level vs. Segment-Level Quantitative Metrics for Surgical Skill Assessment Surg Educ 2016
Relative Hidden Markov Models for Video-Based Evaluation of Motion Skills in Surgical Training TPAMI 2015
Automated Assessment of Surgical Skills Using Frequency Analysis MICCAI 2015
Automated objective surgical skill assessment in the operating room from unstructured tool motion in septoplasty IJCARS 2015
Automated Surgical OSATS Prediction From Videos ISBI 2014
Pairwise Comparison-Based Objective Score for Automated Skill Assessment of Segments in a Surgical Task IPCAI 2014
Video Based Assessment of OSATS Using Sequential Motion Textures MICCAIW 2014
Augmenting Bag-of-Words: Data-Driven Discovery of Temporal and Structural Information for Activity Recognition CVPR 2013
String Motif-Based Description of Tool Motion for Detecting Skill and Gestures in Robotic Surgery MICCAI 2013
Robotic Path Planning for Surgeon Skill Evaluation in Minimally-Invasive Sinus Surgery MICCAI 2012
An objective and automated method for assessing surgical skill in endoscopic sinus surgery using eye-tracking and tool-motion data IFAR 2012
Sparse Hidden Markov Models for Surgical Gesture Classification and Skill Evaluation IPCAI 2012
Towards integrating task information in skills assessment for dexterous tasks in surgery and simulation ICRA 2011
Video-based Motion Expertise Analysis in Simulation-based Surgical Training Using Hierarchical Dirichlet Process Hidden Markov Model MMAR 2011
Eye Metrics as an Objective Assessment of Surgical Skill AnnS 2010
Surgical Task and Skill Classification from Eye Tracking and Tool Motion in Minimally Invasive Surgery MICCAI 2010
Task versus Subtask Surgical Skill Evaluation of Robotic Minimally Invasive Surgery MICCAI 2009
Data-derived models for segmentation with application to surgical assessment and training MICCAI 2009
Multitask learning for video-based surgical skill assessment DICTA 2009 Paper

🔄 Surgical Skill Transfer

Title Venue Links
Deep Neural Skill Assessment and Transfer: Application to Robotic Surgery Training IROS 2021 Paper
Adaptive Surgical Robotic Training Using Real-Time Stylistic Behavior Feedback Through Haptic Cues CoRR 2021 Paper
Robot-Assisted Training in Laparoscopy Using Deep Reinforcement Learning IEEE RAL 2019 Paper
Towards Skill Transfer via Learning-Based Guidance in Human-Robot Interaction: An Application to Orthopaedic Surgical Drilling Skill JINT 2020 Paper
Deep Learning-Based Haptic Guidance for Surgical Skills Transfer FiRaA 2021 Paper
Human-robot skills transfer interfaces for a flexible surgical robot CMaPiB 2014 Paper
DESK: A Robotic Activity Dataset for Dexterous Surgical Skills Transfer to Medical Robots RSJ 2019 Paper Code
Transferring Dexterous Surgical Skill Knowledge between Robots for Semi-autonomous Teleoperation CoRaHIC 2019 Paper

⏱ Surgical Phase Recognition

Title Venue Links
Self-supervised representation learning for surgical activity recognition IJCARS 2021 Paper
Surgical Workflow Anticipation Using Instrument Interaction MICCAI 2021 Paper
Comparative Validation of Machine Learning Algorithms for Surgical Workflow and Skill Analysis with the HeiChole Benchmark Arxiv 2021 Paper
OperA: Attention-Regularized Transformers for Surgical Phase Recognition MICCAI 2021 Paper
Temporal Memory Relation Network for Workflow Recognition from Surgical Video TMI 2021 Paper Code
Multi-Task Temporal Convolutional Networks for Joint Recognition of Surgical Phases and Steps in Gastric Bypass IJCARS 2021 Paper
Train one, Classify one, Teach one"–Cross-surgery transfer learning for surgical step recognition MIDL 2021 Paper
TeCNO: Surgical Phase Recognition with Multi-Stage Temporal Convolutional Networks MICCAI 2020
Automated laparoscopic colorectal surgery workflow recognition using artificial intelligence: Experimental research IJS 2020 Paper
Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysis MIA 2020 Paper Code
LRTD: Long-Range Temporal Dependency based Active Learning for Surgical Workflow Recognition IJCARS 2020 Paper Code
Assisted phase and step annotation for surgical videos IJCARS 2020
Accurate Detection of Out of Body Segments in Surgical Video using Semi-Supervised Learning MIDL 2020 Paper
Impact of data on generalization of AI for surgical intelligence applications SciReports 2020 Paper
Hard Frame Detection and Online Mapping for Surgical Phase Recognition MICCAI 2019
MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation CVPR 2019
Learning from a tiny dataset of manual annotations: a teacher/student approach for surgical phase recognition IPCAI 2019 Paper
Machine and deep learning for workflow recognition during surgery MITAT 2019
Assessment of automated identification of phases in videos of cataract surgery using machine learning and deep learning techniques JAMA 2019 Paper
Multitask learning of temporal connectionism in convolutional networks using a joint distribution loss function to simultaneously identify tools and phase in surgical videos Arxiv 2019 Paper
SV-RCNet: Workflow Recognition from Surgical Videos using Recurrent Convolutional Network TMI 2018 Paper Code
DeepPhase: Surgical Phase Recognition in CATARACTS Videos MICCAI 2018
Surgical activity recognition in robot-assisted radical prostatectomy using deep learning MICCAI 2018 Paper
Knowledge transfer for surgical activity prediction IJCARS 2018 Paper
Temporal coherence-based self-supervised learning for laparoscopic workflow analysis MICCAIW 2018 Paper
“Deep-Onto” network for surgical workflow and context recognition IJCARS 2018 Paper
Less is More: Surgical Phase Recognition with Less Annotations through Self-Supervised Pre-training of CNN-LSTM Networks Arxiv 2018
EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos TMI 2017
Unsupervised temporal context learning using convolutional neural networks for laparoscopic workflow analysis Arxiv 2017 Paper
System events: readily accessible features for surgical phase detection IJCARS 2016 Paper
Automatic data-driven real-time segmentation and recognition of surgical workflow IJCARS 2016 Paper
MICCAI Workflow Challenge: Convolutional neural networks with time smoothing and Hidden Markov Model for video frames classification Arxiv 2016
Automatic phase prediction from low-level surgical activities IJCARS 2015 Paper
Lapontospm: an ontology for laparoscopic surgeries and its application to surgical phase recognition IJCARS 2015 Paper
Real-time segmentation and recognition of surgical tasks in cataract surgery videos TMI 2014 Paper
Statistical modeling and recognition of surgical workflow MIA 2012
A framework for the recognition of high-level surgical tasks from video images for cataract surgeries TBE 2012 Paper
An application-dependent framework for the recognition of high-level surgical tasks in the OR TBE 2011
Modeling and Segmentation of Surgical Workflow from Laparoscopic Video MICCAI 2010

🤖 Surgical Gesture Recognition

Title Venue Links
Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery ICRA 2021 Paper
Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Search ICRA 2020 Paper
Multi-Task Recurrent Neural Network for Surgical Gesture Recognition and Progress Prediction ICRA 2020 Paper
Automated surgical activity recognition with one labeled sequence MICCAI 2019 Paper
Using 3d convolutional neural networks to learn spatiotemporal features for automatic surgical gesture recognition in video MICCAI 2019 Paper
Segmenting and classifying activities in robot-assisted surgery with recurrent neural networks IJCARS 2019 Paper
Deep Reinforcement Learning for Surgical Gesture Segmentation and Classification MICCAI 2018 Paper Code
Unsupervised learning for surgical motion by learning to predict the future MICCAI 2018
Surgical motion analysis using discriminative interpretable patterns AIM 2018 Paper
A dataset and benchmarks for segmentation and recognition of gestures in robotic surgery TBE 2017
Temporal Convolutional Networks for Action Segmentation and Detection CVPR 2017
Temporal convolutional networks: A unified approach to action segmentation ECCVW 2016
Recognizing surgical activities with recurrent neural networks MICCAI 2016
Segmental spatiotemporal cnns for fine-grained action segmentation ECCV 2016
Learning convolutional action primitives for fine-grained action recognition ICRA 2016
Unsupervised surgical data alignment with application to automatic activity annotation ICRA 2016 Paper
An improved model for segmentation and recognition of fine-grained activities with application to surgical training tasks WACV 2015
Learning shared, discriminative dictionaries for surgical gesture segmentation and classification MICCAIW 2015
JHU-ISI gesture and skill assessment working set (JIGSAWS): A surgical activity dataset for human motion modeling MICCAIW 2014
Surgical gesture segmentation and recognition MICCAI 2013
Surgical gesture classification from video and kinematic data MIA 2013 Paper
Sparse hidden markov models for surgical gesture classification and skill evaluation IPCAI 2011
Data-derived models for segmentation with application to surgical assessment and training MICCAI 2009
Automatic detection and segmentation of robot-assisted surgical motions MICCAI 2005

🔧 Surgical Instrument Recognition / Localization

Title Venue Links
ARAS-Farabi Experimental Framework for Skill Assessment in Capsulorhexis Surgery ICRoM 2021 Paper
Surgical instrument tracking for capsulorhexis eye surgery based on siamese networks ICRoM 2022 Paper
A Deep Dive Into Capsulorhexis Segmentation: From Dataset Creation to SAM Fine-tuning ICRoM 2023 Paper
hSDB-instrument: Instrument Localization Database for Laparoscopic and Robotic Surgeries MICCAI 2021 Paper
Prototypical Interaction Graph for Unsupervised Domain Adaptation in Surgical Instrument Segmentation MICCAI 2021 Paper
Co-Generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data MICCAI 2021 Paper
Evaluation of Artificial Intelligence–Based Intraoperative Guidance Tools for Phacoemulsification Cataract Surgery JAMA 2022 Paper
One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Video ICRA 2021 Paper
A Kinematic Bottleneck Approach For Pose Regression of Flexible Surgical Instruments directly from Images ICRA 2021 Paper
Simulation-to-real domain adaptation with teacher–student learning for endoscopic instrument segmentation IJCARS 2021 Paper
Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery MICCAI 2020 Paper
Unsupervised Surgical Instrument Segmentation via Anchor Generation and Semantic Diffusion MICCAI 2020 Paper Code
Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Video MICCAI 2020 Paper Code
Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery MICCAI 2020 Paper
BARNet: Bilinear Attention Network with Adaptive Receptive Field for Surgical Instrument Segmentation Arxiv 2020
Multi-Task Recurrent Convolutional Network with Correlation Loss for Surgical Video Analysis MIA 2020 Paper Code
Real-time surgical needle detection using region-based convolutional neural networks IJCARS 2020 Paper
Learning Representations of Endoscopic Videos to Detect Tool Presence Without Supervision ML-CDS 2020 Paper Code
Incorporating Temporal Prior from Motion Flow for Instrument Segmentation in Minimally Invasive Surgery Video MICCAI 2019 Paper Code
Weakly supervised convolutional LSTM approach for tool tracking in laparoscopic videos IJCARS 2019
Self-supervised surgical tool segmentation using kinematic information ICRA 2019
Learning Where to Look While Tracking Instruments in Robot-assisted Surgery MICCAI 2019
Patch-based adaptive weighting with segmentation and scale (PAWSS) for visual tracking in surgical video MIA 2019 Paper
CATARACTS: Challenge on automatic tool annotation for cataRACT surgery MIA 2019
2017 robotic instrument segmentation challenge Arxiv 2019
Multitask learning of temporal connectionism in convolutional networks using a joint distribution loss function to simultaneously identify tools and phase in surgical videos Arxiv 2019 Paper
U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instruments from Laparoscopic Images EMBC 2019
RASNet: Segmentation for tracking surgical instruments in surgical videos using refined attention segmentation network EMBC 2019
CFCM: Segmentation via coarse to fine context memory MICCAI 2018
Monitoring tool usage in surgery videos using boosted convolutional and recurrent neural networks MIA 2018 Paper
3D Pose Estimation of Articulated Instruments in Robotic Minimally Invasive Surgery TMI 2018 Paper
Exploiting the potential of unlabeled endoscopic video data with self-supervised learning IJCARS 2018 Paper
Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery Arxiv 2018 Paper
Weakly-supervised learning for tool localization in laparoscopic videos MICCAIW 2018
Automatic instrument segmentation in robot-assisted surgery using deep learning ICMLA 2018
Concurrent segmentation and localization for tracking of surgical instruments MICCAI 2017
Simultaneous recognition and pose estimation of instruments in minimally invasive surgery MICCAI 2017 Paper
Toolnet: holistically-nested real-time segmentation of robotic surgical tools IROS 2017
Real-time localization of articulated surgical instruments in retinal microsurgery MIA 2016
Combined 2D and 3D tracking of surgical instruments for minimally invasive and robotic-assisted surgery IJCARS 2016 Paper
Detecting surgical tools by modelling local appearance and global shape TMI 2015
Surgical Tool Tracking and Pose Estimation in Retinal Microsurgery MICCAI 2015 Paper
Toward Detection and Localization of Instruments in Minimally Invasive Surgery TBE 2013 Paper
Unified detection and tracking of instruments during retinal microsurgery TPAMI 2013 Paper
Data-driven visual tracking in retinal microsurgery MICCAI 2012 Paper

🏥 Surgical Scene / Anatomy Recognition

Title Venue Links
Artificial Intelligence for Surgical Safety: Automatic Assessment of the Critical View of Safety in Laparoscopic Cholecystectomy using Deep Learning AnnS 2020 Paper
EasyLabels: weak labels for scene segmentation in laparoscopic videos IJCARS 2019 Paper
Kidney edge detection in laparoscopic image data for computer-assisted surgery IJCARS 2019 Paper
Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy TBE 2018 Paper
Long Term Safety Area Tracking (LT-SAT) with online failure detection and recovery for robotic minimally invasive surgery MIA 2018 Paper

👥 Human / Activity in Operating Rooms

Title Venue Links
Self-supervision on Unlabelled OR Data for Multi-person 2D/3D Human Pose Estimation MICCAI 2020 Paper
Automatic Operating Room Surgical Activity Recognition for Robot-Assisted Surgery MICCAI 2020 Paper
Privacy-Preserving Human Pose Estimation on Low-Resolution Depth Images MICCAI 2019 Paper
Face Detection in the Operating Room: Comparison of State-of-the-art Methods and a Self-supervised Approach IPCAI 2019 Paper
A Multi-view RGB-D Approach for Human Pose Estimation in Operating Rooms WACV 2017 Paper
Pictorial Structures on RGB-D Images for Human Pose Estimation in the Operating Room MICCAI 2015 Paper
Data-driven Spatio-temporal RGBD Feature Encoding for Action Recognition in Operating Rooms IJCARS 2015 Paper
Video based activity recognition in trauma resuscitation FG 2013 Paper

Video-Based Navigation in Surgery

Title Venue Links
Autonomously navigating a surgical tool inside the eye by learning from demonstration ICRA 2020 Paper
Evaluation and stability analysis of video-based navigation system for functional endoscopic sinus surgery on in vivo clinical data TMI 2018 Paper
Endoscopic navigation in the absence of CT imaging MICCAI 2018 Paper
Global rigid registration of CT to video in laparoscopic liver surgery IJCARS 2018 Paper
Intelligent viewpoint selection for efficient CT to video registration in laparoscopic liver surgery IJCARS 2017 Paper
Rendering-based video-CT registration with physical constraints for image-guided endoscopic sinus surgery ISOP 2015 Paper
A system for video-based navigation for endoscopic endonasal skull base surgery TMI 2011 Paper

Others

Title Venue Links
Surgical Instruction Generation with Transformers MICCAI 2021 Paper
SurgeonAssist-Net: Towards Context-Aware Head-Mounted Display-Based Augmented Reality for Surgical Guidance MICCAI 2021 Paper
E-DSSR: Efficient Dynamic Surgical Scene Reconstruction with Transformer-based Stereoscopic Depth Perception MICCAI 2021 Paper
Learning Domain Adaptation with Model Calibration for Surgical Report Generation in Robotic Surgery ICRA 2021
A Computer Vision Platform to Automatically Locate Critical Events in Surgical Videos AnnS 2021 Paper
Ethical implications of AI in robotic surgical training: A Delphi consensus statement EUF 2021 Paper
Future Frame Prediction for Robot-assisted Surgery IPMI 2021 Paper
Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge Arxiv 2021 Paper
Offline identification of surgical deviations in laparoscopic rectopexy AIM 2020 Paper
Recognition of Instrument-Tissue Interactions in Endoscopic Videos via Action Triplets MICCAI 2020 Paper
Orientation Matters: 6-DoF Autonomous Camera Movement for Minimally Invasive Surgery Arxiv 2020 Paper
Future-State Predicting LSTM for Early Surgery Type Recognition TMI 2019 Paper
Dense depth estimation in monocular endoscopy with self-supervised learning methods TMI 2019
Real-time identification of blood regions for hemostasis support in laparoscopic surgery SIVP 2019 Paper
RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotations TMI 2018 Paper
Learning to see forces: Surgical force prediction with rgb-point cloud temporal convolutional networks MICCAIW 2018 Paper
Deep Neural Networks Predict Remaining Surgery Duration from Cholecystectomy Videos MICCAI 2017 Paper
Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery IJCARS 2017 Paper
Surgical Soundtracks: Towards Automatic Musical Augmentation of Surgical Procedures MICCAI 2017 Paper
Distinguishing surgical behavior by sequential pattern discovery JBI 2017 Paper
Finding discriminative and interpretable patterns in sequences of surgical activities AIM 2017 Paper
Automatic matching of surgeries to predict surgeons’ next actions AIM 2017 Paper
Query-by-example surgical activity detection IJCARS 2016 Paper
Classification Approach for Automatic Laparoscopic Video Database Organization IJCARS 2015 Paper
Work domain constraints for modelling surgical performance IJCARS 2015 Paper
Smoke detection in endoscopic surgery videos: a first step towards retrieval of semantic events IJMRCAS 2015 Paper
Fisher Kernel Based Task Boundary Retrieval in Laparoscopic Database with Single Video Query MICCAI 2014 Paper
Non-linear temporal scaling of surgical processes AIM 2014 Paper
Surgical process modelling: a review IJCARS 2014 Paper
Real-time dense stereo reconstruction using convex optimisation with a cost-volume for image-guided robotic surgery MICCAI 2013 Paper
Vision-based proximity detection in retinal surgery TBE 2012 Paper
Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures IJCARS 2012 Paper
Similarity metrics for surgical process models AIM 2012 Paper

About

Awesome Image-guided Surgery

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published