How Interactive Mixed Reality Training can help fight COVID-19

As the COVID-19 pandemic grows in the United States and across the world, many people in your life will be impacted by the disease, whether themselves or a loved one falls ill. Healthcare workers are already struggling to deal with COVID-19 patients: while many people show little to no symptoms and can recover at home, at-risk patients often require a ventilator and intense care, and are exponentially multiplying each day.

States such as New York and Illinois have urged retired healthcare workers and other qualified medical professionals to return and assist frontline medical staff with the expected influx of patients. These volunteers will need to be re-trained to care for patients in hospitals that are quickly reaching capacity, and where up to 40% of the current medical staff may fall ill. The University of Illinois at Urbana-Champaign and OSF HealthCare recently developed an Interactive Mixed Reality (IMR) training tool for sepsis, a disease that can be difficult to diagnose and is often fatal if not treated in a timely manner. IMR encompasses both virtual and augmented reality (VR and AR), merging real and virtual worlds to produce new learning environments. This tool makes it easier for healthcare providers to diagnose by directing them through sepsis prevention protocol with a 360-degree video recording and annotation, proving the efficacy of new simulated environments in training novice medical professionals. With the COVID-19 pandemic growing, IMR training could easily be adapted to train these volunteer medical professionals in quickly and correctly providing patients the aid they need to survive.

“Simulation technologies such as virtual and augmented reality are becoming more common in medical training,” said T. Kesh Kesavadas, Director of the Health Care Engineering Systems Center (HCESC) at the University of Illinois. VR and AR are a major focus of HCESC, a research center that fosters collaboration between engineers and physicians. Other areas of focus for HCESC are surgical robotics, health data analytics, and smart health. Kesavadas states, “Our center is at the forefront of developing these technologies as well as software that makes it possible for faculty without programming skills to create IMR scenarios.”

HCESC’s goal is that increasing usability of VR and AR technologies for instructors will make them more widespread in training novice medical professionals and improving existing healthcare processes. These technologies allow students to practice techniques more thoroughly prior to practicing on a simulated or actual patient but are not easy to create; lack of effective software for instructors to easily develop VR curriculum content is a hurdle that HCESC is working diligently to overcome.

In partnership with OSF HealthCare, HCESC has created a software platform that simplifies the creation of IMR. The sepsis protocol training referenced above is comprised of three parts: an integrated, 360-degree video recording of a clinical encounter to provide first-person perspective, rich annotated content, and an assessment questionnaire. “An important part of simulation is including real-world parameters: patient physiology, emotions, and clinical team behaviors all contribute to how successful an encounter might be. Our software platform allows instructors to easily include all these parameters,” Kesavadas said.

“OSF HealthCare is testing VR as a replacement for traditional simulation training for nurses. VR-based software has made this training accessible to nurses in a timely manner at their workstation,” added John Vozenilek, M.D., Vice President and Chief Medical Officer for Innovation and Digital Health at Jump Trading Simulation and Education Center at OSF HealthCare. “We are planning to develop a comprehensive set of curriculum for nursing staff and other health professionals using the new IMR technology.”

HCESC’s sepsis prevention education scenario demonstrates the potential of enhancing simulated medical training by accelerating clinical exposure for novice students. “We conducted an IRB-approved study with 28 novice students to evaluate its efficacy, and proudly concluded that our IMR software is a usable technology,” Kesavadas said. The participants provided feedback by answering demographics, NASA-TLX, and system usability scale questionnaires. “It’s a step towards improving VR-based education content development.”

The portable nature of IMR training allows for quick and easy adoption, ultimately benefitting patients from large hospital complexes to hospitals with limited resources. It would be extremely beneficial in re-training retired/volunteer medical professionals to assist with the COVID-19 pandemic. The Health Care Engineering Systems Center at Illinois welcomes any collaboration with hospitals wishing to use this technology in training their volunteers. COVID-19 will continue to test our healthcare system, but in doing so will result in brilliant solutions to challenges that face our world.