PyModalSurgical. An image-space modal analysis library for surgical videos: generating haptic and visual feedback.

[mikelitu]

🎒 Poster submission

Welcome Poster submission for 🎒 Open-Source Software for Surgical Technologies 🎉

Personal Details

• First author:
  • family-names: De Iturrate Reyzabal
  • given-names: Mikel
  • email: [email protected]
  • orcid: 0000-0001-9503-8834
  • Department/Faculty: Biomedical Engineering and Imaging Sciences
  • affiliation: King's College London
• Second author:
  • family-names: Ourselin
  • given-names: Sebastien
  • email: [email protected]
  • Department/Faculty: Biomedical Engineering and Imaging Sciences
  • affiliation: King's College London
• Third author:
  • family-names: Liu
  • given-names: Hongbin
  • email: [email protected]
  • Department/Faculty: Biomedical Engineering and Imaging Sciences
  • affiliation: King's College London / Centre for Artificial Intelligence and Robotics (Hong Kong)

Which subtheme most closely connects to your poster?

• Open-source software libraries and frameworks,
• Medical and surgical software innovations,
• Sustainability in open-source software,
• Case studies showcasing novel applications and combinations of existing software,
• Protocols for managing clinical data in computer-assisted software for surgical technologies, and
• Project summaries, methodological approaches, research findings, and initiatives related to Open-Source Software for Surgical Technologies.
• Other (please add topic).

Poster title

PyModalSurgical. An image-space modal analysis library for surgical videos: generating haptic and visual feedback.

Briefly describe your poster proposal

We present a new open source Python library to apply image-space modal analysis to surgical videos called PyModalSurgical. This repository collects and deploys the necessary tools to calculate the frequency response of deforming tissue directly from surgical videos. This method is known as Modal Analysis and it is widely used in many areas of engineering for the characterization of mechanical systems. To perform such analysis, we use a combination of well-known and widely used foundation models such as MiDaS or RAFT. The resultant mode shapes provide a physically relevant solution to the mass-spring system modelling soft tissue small deformations and represent the basis of motion of the tissue in the image-space.

As a result, this repository could potentially be used to generate force labels for self-supervised training of neural network models; estimate forces in remote areas far from force sensors or contacts and generate force maps to evaluate the performance of surgeons purely based on the information captured from video. Additionally, we provide the possibility to generate visually realistic direct manipulation interactive demos from video sources, to test both the reconstructed force and visual feedback. Github project.

Link to the poster: https://docs.google.com/presentation/d/11Ge2ExigHWTBvwJ512Jssetc-URY8HnfEdl3yuWVD4I/edit?usp=sharing

[mxochicale]

Dear @mikelitu
Thank you for your submission! We are pleased to inform you that your poster has been accepted.

We have extended the poster submission deadline to June 14 to encourage additional submissions. As a result, we will provide further details on submitting posters to Zenodo on June 17. In the meantime, please feel free to start designing your poster in either A0 or A1 format for its presentation on June 28. For additional information please add comments in the current issue or contact Eva Herbst at [email protected] and Miguel Xochicale [email protected]

Thanks, @evaherbst @thompson318, and @mxochicale

[mxochicale]

Dear @mikelitu

Please refer to Zenodo DOI's for accepted poster for further details on Zenodo submission! We have sent you an email with further details of our workshop, please do not hesitate to reach via this issue or email if there is anything else we can help!

Looking forward to meeting you!
Thanks, @evaherbst @thompson318, and @mxochicale