Assistant professors Taiwoo Park of ComArtSci and Mi Zhang of the College of Engineering are bringing the power of technology to day-to-day life with JARVIS, a computational system named after Tony Stark’s Iron Man suit from the Marvel film series The Avengers. With JARVIS, Park and Zhang have brought together ubiquitous computing and mixed reality (MR) technology to collect research and enhance daily routines like hitting the gym.
“[The purpose of] this research is to let you experience MR anytime, anywhere in your life,” said Park. “We are creating an interactive system that automatically understands what is happening around you—ubiquitous computing—and draws a virtual world nicely blended with your physical world in front of your eyes— mixed reality.”
Brains and Brawns
The researchers studied how JARVIS’ MR system operates in a fitness setting. Participants attached a sensory device to exercise machines at the gym, which collected data. JARVIS then analyzed and responded to a user’s movement during a workout, producing an MR scene for the user. Once a user begins working on an exercise machine, JARVIS will detect the type of exercise and create a personalized exercise coach within two reps. The research team’s sample app for seated abdominal training also showed that the app can help users better activate target muscle groups during a workout by showing the user’s virtual self in the MR world.
“JARVIS had over 99% accuracy in its human activity recognition system, meaning users minimized inconvenience,” said Park. “You may feel 90% accuracy is enough, but it really isn’t, as it means about once every 10 times of exercise you will get a wrong support.”
This means that JARVIS is keenly attuned to recognize the movements of the person exercising to produce a more efficient workout experience.
The Power of Collaboration
ComArtSci supported the development of the research through the interactive media studio, which Park described as the “powerhouse” behind the idea. He also expressed his appreciation for the college’s culture of facilitating university-wide collaboration. The research was conducted with support from Trifecta Initiative funding as well as startup funding from ComArtSci and the College of Engineering.
Zhang said the research is a natural combination between his research on intelligent sensor systems and Park’s research on virtual reality and augmented reality systems.
Park and Zhang’s study has been published in the Association for Computing Machinery’s Interactive, Mobile, Wearable and Ubiquitous Technologies and GetMobile.
For the next step in their study, Park and Zhang are looking to expand the context in which JARVIS can be used outside of the gym. They’re working with external collaborators to explore how to create more variety in ubiquitous computing and MR applications.
“MR has been gaining huge attention from both industry and academia recently with the advances of virtual and augmented reality systems,” Park said. “There is going to be a number of great opportunities for interactive media and application designers and researchers to create innovative, life-immersive services using these technologies.”
By Kristina Pierson