Amviewlate
Overview
Amviewlate is an augmented reality prototype designed to help people with Parkinson's Disease navigate environments using voice recognition and visual cues. Amviewlate was selected as a DevPost Staff Pick from hundreds of hackathon entries.
Project Details
Role: Research and Prototyping
Team: Pierre Amelot, Sara Stalla
Methods: Interviews, Literature Review, Paper Prototyping, VR Prototyping
Tools: Blender, Unity, Vuforia, Google Cardboard, Android Speech Recognizer
The problem
More than 10 million people worldwide are living with Parkinson's disease. Parkinson’s patients have trouble controlling their movements from reaching for objects to simply walking across the room. These problems lead to loss of independence and reduced quality of life: sufferers often avoid activities outside of the home, leading to isolation and depression.
At the start of the Hackathon, we asked ourselves, how can we empower people with Parkison's to feel more comfortable engaging with the world around them?
Process
To begin, we conducted several guerilla interviews with friends and family who either suffered from, or knew people who suffered from Parkinson's Disease. Our guiding insight came after watching Mileha Soneji's TED talk: Simple hacks for life with Parkinson's. Through this and other research, we discovered that many of the limitations experienced by people with Parkinson’s disease can be overcome with perceptual tricks.
While the pace of walking on a flat surface is staggered and interrupted, the continuous motion of using a staircase, or even the illusion of a staircase, can enable a person with Parkinson’s to walk with a regular gait. And while many encounter difficulty extending an arm to interact with an object, the introduction of a moving target provides an active stimulus that prompts a response. In one video interview, Tom, a 40 year old man suffering from Parkinson's, said: "If you get me up a flight of stairs I'd have no problem at all. If someone chucked a ball at me, my hand would go straight out. And yet if you asked me to put my arm out, it would go so slowly."
Paper Prototyping and 3D design
What if we could take this painted staircase illusion and other visual cues and apply them anywhere in the world? So we set out to create an augmented reality solution focusing on two motion issues occurring in people with Parkinson's: "freezing of gait," an ambulatory disturbance that results in slow, shuffled walking, and "dyskinesia," the impairment of voluntary movement, particularly without stimuli to trigger reflexes.
We used paper markers to prototype the ideal motion cues for walking. Based on research and prototyping, the final project used three visual cues: staircase for walking, rotating footprints for turning, and a flying baseball for grabbing objects. While we weren't able to co-design the experience with people with Parkinson's, we used this stage to work out central usability questions, such as, what is a natural speed and direction of cues for turning? Through this prototyping, we also realized that voice command would be a huge asset to trigger different illusions, so motion could be as seamless as possible. Once we determined this, I searched and altered 3D assets and scripts for the foot, ball, and stair illusions using the Unity Store and Blender while Pierre got to work getting the AR and voice recognition up and running.
Three cues
Demo
Amviewlate was developed over 48 Hours for Google Cardboard at the 2016 ArchHacks HealthTech Hackathon. Watch Amviewlate in action, with image markers and voice recognition used to trigger three types of cues found to help people with Parkinson's disease: a staircase illusion for walking, foot rotation for turning, and a flying ball for grabbing objects.