Review Article Open Access

Remote Surgeon Hand Motion and Occlusion Removal in Mixed Reality in Breast Surgical Telepresence: Rural and Remote Care

Krishna Shakya1, Suman Khanal1, Abeer Alsadoon1, P.W.C. Prasad1, Jeremy Hsu2, Anand Deva2, Manoranjan Paul1 and A. Elchouemi3
  • 1 School of Computing and Mathematics, Charles Sturt University, Sydney, Australia
  • 2 Faculty of Medicine and Health Sciences, Macquarie University, Australia
  • 3 Walden University, Australia

Abstract

Mixed reality in surgical telepresence has not been practically implemented as yet and research done, until now, is only theoretical. Aiming to implement a proposed new solution by merging virtual and augmented realities and bringing about improvements by removing occlusion and the noise caused by hand motions during surgery. The proposed system consists of an extended Camshift algorithm for the detection of the remote expert surgeon’s hand during surgery and enhancement of an image synthesization algorithm to remove the occlusion by including two Red-Green-Blue-Depth (RGBD) cameras. The experimental results from 10 breast samples reduce the average accuracy error of the video frame image overlay from 1.44 mm to 1.28 mm with average processing time enhanced to 76 sec compared to the state-of-the-art method. The proposed system provides a satisfactory outcome in terms of accuracy and operating time, which allows surgeons in local and remote sites to collaborate more effectively.

American Journal of Applied Sciences
Volume 15 No. 11, 2018, 497-509

DOI: https://doi.org/10.3844/ajassp.2018.497.509

Submitted On: 22 September 2018 Published On: 11 January 2019

How to Cite: Shakya, K., Khanal, S., Alsadoon, A., Prasad, P., Hsu, J., Deva, A., Paul, M. & Elchouemi, A. (2018). Remote Surgeon Hand Motion and Occlusion Removal in Mixed Reality in Breast Surgical Telepresence: Rural and Remote Care. American Journal of Applied Sciences, 15(11), 497-509. https://doi.org/10.3844/ajassp.2018.497.509

  • 2,693 Views
  • 1,818 Downloads
  • 7 Citations

Download

Keywords

  • Virtual Reality
  • Mixed Reality
  • Diminished Reality
  • Augmented Reality
  • Telesurgery
  • Visualization
  • Tele Collaboration
  • Surgical Telepresence