Research and development in Collaborative Virtual Environments, Virtual Reality, Augmented Reality and Telemanipulation and their application in distance diagnosis, medical procedure planning and training.

Virtual Reality environments are especially useful for training applications, as a user is exposed to a synthetic world that may mimic the real experience of the procedure being simulated. Just as an aircraft pilot learns to fly the aircraft and acquires flight experience in simulators, surgeons could learn how to perform a given surgical procedure, as well as acquiring surgical experience through a variety of surgery situations that the surgeon has to handle. Na additional advantage has to do with the fact that a given user may be subjected to a well-designed pool of test cases that varies from the most common every-day procedures to extremely rare and unique, which allows a surgeon to gain a more uniform experience level when compared with his or her pairs. Right now medics learn by performing the procedures in-vivo, which ends up limiting the experience acquired to those cases that happened to show up while the training was ongoing).
Another topic in which virtual reality is very useful is in rehabilitation, which includes physical rehabilitation, as well as phobia treatment. With regard to physical rehabilitation, it is hard to get the user excited about, involved with physiotherapy exercises, especially youngsters, and aged patients. A well-designed VR system will require the user to perform exactly the required exercises in order for the user to obtain success in a simulated game. That allows patients to have more interest in an engaging physiotherapy session. An example would be a user who has partially lost the motion of one side of the body, due to a brain stroke incident, who is encouraged to perform ever-increasing motion of that side of the body while engaged in a VR simulation that attracts interest of the patient.
On the other hand, augmented reality allows users to gain access to additional information that is attached to a real-life scene. For example, consider a surgeon that gets access to support imagery, such as X-rays, 3D models obtained out of CAT scans, which may even be superimposed over the real structure, while performing the surgery (let us consider a surgery in the head, where the surgeon can see the blood vessels and other structures of interest superimposed over the patient head, similar to an “X-ray vision”).
With regard to Virtual Reality and Augmented Reality, a number of simulation prototypes are expected to be developed during the term of this project. Some prototypes focus on surgical simulations (out of which we can feature a catheterism procedure simulation) whilst others on rehabilitation.
Activities:

1. Design and development of a tactile device for catheterism simulation.
2. Design and development of a catheterism simulator.
3. Design and development of a videolaparoscopy simulator (gallbladder removal) using the LapVR hardware.
4. Design and development of a motor rehabilitation using the CyberForce / CyberGrasp / CyberGlove system.
5. Development of a simulation of venipuncture procedure.
6. Development of a simulation of endodontic procedure.
7. Development of augmented reality based surgical support system.
8. Development of support system for multimodal interaction content retrieval diagnosis.

Goals:

1. By end of 2017: Development of tactile system for catheterism.
2. By the end of 2019: Development of catheterism procedure simulation.
3. By the end of 2019: Development of videolaparoscopy simulation prototype.
4. By the end of 2021: Development of laparoscopy based gallbladder removal simulation.
5. By the end of 2019: Development of prototype of motor-rehabilitation simulation based on the CyberForce/ CyberGrasp/CyberGlove system.
6. By the end of 2021: Development of motor-rehabilitation simulation based on the CyberForce/ CyberGrasp/CyberGlove system.
7. By the end of 2021: Development of surgical simulation environment based on the CyberForce/ CyberGrasp/CyberGlove system.
8. By the end of 2019: Development of venipuncture procedure simulation.
9. By the end of 2019: Development of endodontic procedure simulation.
10. By the end of 2021: Development of augmented reality based surgical support system.
11. By the end of 2019: Development of support system for multimodal interaction content retrieval diagnosis.

Impacting results:

• The catheterism simulator allows that someone practice the procedure, acquiring experience through the simulator, where even rare cases can be simulated, as desired by the session instructor. Such model allows a uniform medical training, moving away from the current setup, where the practice depends on the real medical cases that happen to show up.
• The videolaparoscopy simulator, much like the catheterism simulator, allows that some standardization in the training sequence is established, where a diverse set of reference cases, even those extremely rare, are exposed to the user.
• The venipuncture simulator allows the users to improve their technique before they need to interact with real case scenarios. The same happens with the endodontic simulator.
• The augmented-reality based surgical support system allows a surgeon to access data that is relevant to the procedure, in real time and without losing visual contact with the real-life scenario. The information presented may include radiographies, 3D models that come from CAT scans and alike.
• The physical rehabilitation system allows its users to have a more engaging experience whilst performing the needed physiotherapy. Such setup increases the quality of life of the users, as they will have the benefit of the physiotherapy without the hurdle of the repetitive exercises, which will be performed naturally in the engaging VR environment.
• An additional result achieved if the reduction of costs in training, as that can be carried out in the simulation, as many times and required, reducing the need for in-vivo training.