Input devices in the video game industry have evolved over the decades from being simple joysticks to sophisticated controllers that emit haptic feedback. In Enabled Play, a new piece of assistive tech created by self-taught developer Alex Dunn, users get a different kind of input: facial expressions. Companies like Microsoft have sought to expand accessibility through adaptive controllers and accessories. His new device takes these efforts even further by translating the head movements of the users, facial expressions, real-time speech and other non-traditional input methods into mouse clicks, keystrokes and thumbstick movements.
The device has raised eyebrows — quite literally. “Enabled Play is a device that learns to work with you — not a device you have to learn to work with,” says Dunn who lives in Boston. Speaking via Zoom, the 26-year-old said he created Enabled Play to allow everyone — including his younger brother with a disability — to interface with technology naturally and intuitively. At the beginning of the coronavirus pandemic, the only thing he and his New Hampshire-based brother approximately 70 miles apart, had been game. “That’s when I started to see first-hand the challenges that he had faced and the limitations that games had for people with disability.”
This geographer helps video game developers avoid angering countries
Nine years ago, Dunn dropped out of Worcester Polytechnic Institute to become a full-time software engineer. He began researching and developing Enabled Play two and a half years ago. It initially proved challenging as most speech-recognition programs lagged in response time. “I built prototypes with voice commands. When I started talking to people who were deaf and suffered a range of disabilities, I found that voice commands didn’t cut it.” That’s when he started thinking outside the box.
Having already built Suave Keys, a voice-powered program for gamers with disabilities, he created Snap Keys, an extension that turns a user’s Snapchat lens into a controller when playing games like Call of Duty, Fall Guys and Dark Souls. In 2020, he won two awards for his work at Snap Kit Developer Challenge, a competition among third-party app creators to innovate Snapchat’s developer tool kit. With Enabled Play, Dunn takes accessibility to the next level.
With a wider variety of inputs, users can connect the assistive device equipped with a robust CPU and 8 GB of RAM — to a computer, game console or another device to play games in whatever way works best for them. He also spent time ensuring Enabled Play is accessible to people who are deaf and those who want to use non-verbal audio input, like ‘ooh’ or ‘aah’ to act. Enabled Play’s vowel sound detection model is based on “The Vocal Joystick” that engineers and linguistics experts at the University of Washington had developed in 2006.
“Essentially, it predict the word you are going to say based on what is in the profile rather than trying to assume it could be any word in the dictionary,” he says. “This helps cut through Machine Learning bias by learning more about how the individual speaks and applies it to their desired commands.” The AI-enabled controller takes into account a person’s natural tendencies.
If a gamer wants to set up a jump command every time they open their mouth, Enabled Play would identify that person’s individual resting mouth position and set that as the baseline. In January this year, Enabled Play officially launched in six countries, with its user base extending from the US to the UK, Ghana and Austria. One of his primary goals had been to fill a gap in accessibility – and pricing – compared to other assistive gaming devices. “There are Xbox Adaptive Controller, HORI Flex [for Nintendo Switch] and Tobii which does eye-tracking and stuff like that. Still it wasn’t enough,” he said. Compared to devices that are only compatible with one gaming system or computer at a time, his AI-enabled controller — priced at US$249.99 — supports a combination of inputs and outputs.
Speech therapists say that compared to augmentative and alternative communication (AAC) devices, which are medically essential for people with disabilities, his device offers simplicity. Julia Franklin, a speech-language pathologist at Community School of Davidson in Davidson, North Carolina, introduced students to Enabled Play this summer as it is a better alternative to other AAC devices on the shelf that are often “expensive, bulky and limited” in usability. Several sophisticated AAC systems come in the range of US$6,000 to US$11,500, with low-end eye-trackers running in the thousands. A person can also download AAC apps on their mobile devices from US$49.99 to US$299.99 for the app alone. The Enabled Play device allows individuals to leverage their strengths and movements that are already present.
Internet users have applauded the work and noted asking for accessibility should not equate to asking for an “easy mode” — a misconception often cited by critics of making games more accessible. “This is how you make gaming accessible,” one Reddit user wrote about Enabled Play. “Not by dumbing it down, but by creating mechanical solutions that allow users to have the same experience and accomplish the same feats as [people without disabilities].” Another user who regularly worked with young patients with cerebral palsy speculated Enabled Play “would quite literally change their lives.”
Accessibility consultants are building a more inclusive video game industry
The device is not limited to the gaming sphere. It is also being used in schools to make computer labs more accessible. The rise in remote work and online learning environments brought on by the pandemic, according to Jaipreet Virdi, a historian, author and professor at the University of Delaware, has let the device serve as a model for “inclusive participation” in schools. In several therapy programs in the US, specialists use Enabled Play to track facial expressions and gamify treatment sessions. Alissa McFall, a speech-language pathologist and orofacial mycologist in Sacramento, says it can be used to analyze how a patient’s muscles work to develop customized treatment plans. “The biggest value in using the Enabled Play device is that it can be programmed to read natural communication movements and connect each sound or facial expression to a function that is meaningful to an individual.” Since its launch, Enabled Play has partnered with organizations in the gaming and assistive tech sphere, including Special Effect, Makers Making Change and Microsoft with its Designed for Xbox accessibility partners program. Dunn hopes to soon roll out “virtual devices” which would allow other developers to add Enabled Play’s inputs to their apps. With these additions, a person could use facial expressions and voice commands in Microsoft Word and Adobe Photoshop without buying a separate device.
As developers look for ways to make tech more accessible, he hopes to help drive the change and encourage others to think beyond the typical keyboard and mouse inputs.
“It is my very personal mission to solve these problems. I’m after to build devices that change the human-computer interaction paradigm to one that’s just more inclusive.”
Amanda Florian is a journalist who shuttles between the US and Shanghai