The department of Information Engineering at the University of Pisa coordinated the research and development of the new device, named VOSTARS. The abbreviation stands for 'Video Optical See-Through Augmented Reality surgical System', which is exactly what the device strives to achieve. While wearing the head-mounted display system, surgeons are able to see a patient’s x-ray’s and data in real time, projected onto the patient’s body. This makes it possible to improve surgeries and reduce the amount of time spent in surgery. Reducing the time spent in surgery also means the patients doesn’t have to spent as much time under anaesthetic.
The VOSTARS project received a grant of €3,816,440 from Horizon 2020 via the Photonics Public Private Partnership.
For this project, however, a combination had to be made. Both approaches weren’t suitable for operating on a real patient. OST brings the more naturalistic feel, while VST offers fluid interaction with the world around the user. To achieve this, the researchers had to make a new hybrid from scratch.
VOSTARS aims to have a working prototype of the hybrid device ready for May of 2018. Initially being trialled on a number of procedures to the head, the project hopes to be available to end users in 3 years, with mass production by 2022.
The VOSTARS project received a grant of €3,816,440 from Horizon 2020 via the Photonics Public Private Partnership.
Merging realtime footage with medical data
Before stepping into surgery, the surgeon places the device on his or her head. During surgery, the front facing camera of the VOSTARS system captures what the surgeons sees. The system then 'merges' this footage of reality with the patient’s medical images, from CT, MRI, or 3DUS scans. The surgeon then sees the compiled hybrid image on the surgical ‘dashboard’. The visor also presents a patient’s anaesthetic data, heart rate, body temperature, blood pressure, and breathing rates into the surgeon’s field of vision, in a drive to increase accuracy by focusing on the operation and reduce time by never having to look away. A clinician can move freely while still seeing the patient, the hybrid x-ray image and all of critical data all at once in a surgical 'dashboard' inside the screen.Combining approaches
There are currently two types of AR headsets approaches: video see-through (VST) system and optical see-through (OST). The scientists of VOSTARS combine these approaches to get the benefits of both. In 'optical see-through' (OST) systems, computer-generated images are superimposed on a user's field of vision. The user however does have a direct view of the natural environment. The app Pokémon Go is an example of this approach, where small amounts of the virtual world are added to the real world. With 'video see-through' (VST) systems, the user experiences life through screens. The user is truly submerged in the virtual world. Occolus Rift uses this approach, where the virtual takes over the real world.For this project, however, a combination had to be made. Both approaches weren’t suitable for operating on a real patient. OST brings the more naturalistic feel, while VST offers fluid interaction with the world around the user. To achieve this, the researchers had to make a new hybrid from scratch.
VOSTARS aims to have a working prototype of the hybrid device ready for May of 2018. Initially being trialled on a number of procedures to the head, the project hopes to be available to end users in 3 years, with mass production by 2022.
