Last fall, Greta Van Fleet took the stage at Las Vegas’ Life is Beautiful festival for a show unlike any other.
With ASL music interpreter Amber Galloway and her team perched next to them ready to interpret their lyrics through sign, a crowd of nearly 200 people stood primed to rock out. Half the crowd was deaf, half was not, but for the next hour, you couldn’t tell which was which as they jammed out together to the band’s Led Zeppelin-inspired tunes.
That’s because the crowd was wearing Not Impossible Labs’ latest innovation, an LED-lit wearable tech vest. Equipped with a series of haptic actuators, the Music: Not Impossible device translated the band’s thumping bass, soaring vocals and guitar licks into a range of vibrating pulses sent to 24 points across the body, essentially creating a surround-body experience.
To put it another way, the device translated the communal experience of live music to a community that’s often left out of it.
Based in LA, Not Impossible Labs has spent the last four years building this device. From the start, the team’s goal has been to not only give a person who is deaf a new way to enjoy live music, but also to create a new way for everyone to experience music together.
To get there, it took years of research, experimentation and generating feedback from a community of musicians and people who are deaf.
Lesley Onstott, Not Impossible Labs’ director of projects, broke down how the Music: Not Impossible team created a new way for the Deaf community and non-deaf alike to experience music.
Here are a few tips she picked up on building accessible technology along the way:
-
Make inclusivity, not accessibility, the foundation of your product. Not Impossible Labs set out to create a device that didn’t just enhance the music experience for the deaf community but could be enjoyed by all concertgoers.
-
Understand the why behind the product. Not Impossible Labs grounded its project with a simple vision of a deaf person and a hearing person enjoying the show together. This helped shape the direction of its technology and unified its teams.
-
Do your research and consult other communities to understand their needs. Use that information to make sure what you create meets those needs.
-
Seek feedback from all users, not just people with disabilities. Truly inclusive technology must be an additive experience for all.
-
Open your tech up to an open-source community. Other technologists may have bigger ideas about how to incorporate your product and expand what is possible.
TELLING A STORY CAN UNIFY A PROJECT VISION
Every project at Los Angeles-based Not Impossible Labs begins with what founder Mick Ebeling calls an absurdity. The term is Ebeling’s shorthand to describe challenges a person experiences that underscore a larger societal need.
This can be a skateboarder robbed of his eyesight and thus his ability to skate, a once-adventurous graffiti artist paralyzed and bed-bound due to ALS, or the societal challenge of people going hungry. In each case, Ebeling and his team take it upon themselves to find a solution with technology.
The company has created an eyewriter — a device that relays the movement pattern of a person’s eye to a projector system — to allow the graffiti artist to write on walls again. It’s also built a text messaging platform that connects those lacking food sources and sonic technology that makes it possible to skateboard without eyesight.
When it comes to live music, a person who is deaf often has limited means to join in on the experience. They can either stand close to the speakers to feel the music vibrations or hold a balloon or water glass to feel it. However, in a crowded venue, speakers can be blocked by large crowds and equipment, and holding a glass of water isn’t always easy.
“I find that narrative makes what we’re learning through our research and users relatable.”
So Not Impossible Labs envisioned a world where a person who is deaf can attend a live concert, put on a device and experience the show with the same freedom to move as the people around them.
Onstott said understanding the project through that narrative unified the project. It gave director of technology Daniel Belquer a starting point in the ideation phase, and also allowed Onstott to express the goals of the project to the team as it grew.
“Not everyone on the project gets that firsthand experience of being out in the field, talking to researchers and our users,” Onstott said. “I find that narrative makes what we’re learning through our research and users relatable.”
How do you translate music for those who can’t hear?
Before they could build a tool that would help people who are deaf enjoy music, Not Impossible Labs needed to better understand the science behind music.
To do that, the team turned to Dr. David Putrino, who runs a rehabilitation innovation lab at Mount Sinai. Putrino explained the science behind how the brain engages with music and what tech solutions have been tried.
Since music comes down to vibrations, the auditory cortex isn’t the only part of the brain responsible for interpreting sounds. The hippocampus, cerebellum and amygdala all work together to translate music vibrations into the emotional, physical and memory responses you experience while dancing to Beyonce or grooving to The Grateful Dead.
From a product perspective, Putrino’s research helped Belquer build prototypes that interpreted music for a person who is deaf in a recognizable and effective way through vibrations, Onstott said. They learned that the skin could be just as useful of a music receptacle as the ear.
“We didn’t want to position this as an accessibility tool.”
This helped them start thinking about how to build a product that would enhance the music experience for people who can hear as well. Belquer started with a DIY patchwork of pagers that he connected to an audio file on his laptop to test that theory.
While the motors were blunt and way too loud, it still proved that different instrument sounds could be relayed into vibration. The team then routinely tested the prototypes and sought feedback from several members of the Deaf community. Mandy Harvey, a deaf singer songwriter, played a critical role in trying out early prototypes and explaining what did and didn’t work, Onstott said.
She tried on everything from an 8x8 matrix of actuators to a baseball vest covered in motors, helping Belquer understand that it’s not the number of motors that matters, but the depth of feeling they provided.
“She came in very early in the project,” Onstott said. “She helped us understand how one vibration felt to her, how she interpreted them and where she preferred to experience different parts of the music, like the vocal vibrations and the bass. Her feedback was foundational to what we were doing.”
Meanwhile, American Sign Language music interpreter Amber Galloway held a workshop with the Music Not Impossible team to ensure they understood the experiences of the Deaf community.
“We didn’t want to position this as an accessibility tool,” Onstott said. “We wanted to position it more as an incredible experience that everyone, regardless of level of hearing, will enjoy.”
That feedback and analysis played a critical role in helping the Not Impossible team turn its prototypes into a minimum viable product that they could test.
Field-testing a device dependent on connectivity
After four years of tinkering and repeated tests, the Music Not Impossible wearable device is now on its ninth prototype. The latest version includes haptic actuators inside a wireless vest, with wrist straps and ankle straps that translate sound into vibrations along 24 points of impact.
The vest then connects to the company’s software via a long-range wireless radio signal. The software communicates with the soundboard and translates the music into a range of vibrating pulses that spans five octaves.
With the help of Avnet, an electronic component distributor, the company produced 220 copies of its wearable device. Last year, the company unveiled that device for a Beta test at a Greta Van Fleet show at the Life Is Beautiful music festival in Las Vegas. The show featured a crowd that was half deaf and half hearing-abled to test if the device bridged the concert experience.
In order to reach the Beta phase, the device needed to meet several key criteria, Onstott said. It needed to work in a large setting without the Wi-Fi signals that connect it to the software going out, it needed to be stylish and simple to put on, and it had to be easy to move around in. In essence, it needed to blend in with the concert experience.
After several smaller-scale trials, the Life Is Beautiful show allowed them to test those attributes at scale.
“The reality is that If we want to scale this, the more users who adopt it will mean more venues will get on board, which will mean more shows will become inclusive.”
It worked. When the show started, the team got to watch the crowd dance and bop their way to the band’s Led-Zeppelin-inspired rock.
Afterward, the team polled members of the crowd about the experience. Taking a cue from Galloway’s workshop, they focused on open-ended questions to prevent making assumptions about a person’s experience and ensure they received the right feedback. They asked the concertgoers to describe the experience, how it made them feel and what felt good or confusing.
“We got positive feedback from different folks who were there,” Onstott said. “The reality is that If we want to scale this, the more users who adopt it will mean more venues will get on board, which will mean more shows will become inclusive. Having all of those perspectives at the show was important to us.”
They declared the Beta a success based on the number of attendees who said they enjoyed the experience and said they’d try it again.
Adapting the wearable for new use cases
With a successful beta test in the books, Onstott and the Not Impossible Labs team are now trying to figure out how to expand their technology’s reach.
Onstott has worked closely with music venues in New York, Las Vegas and Los Angeles to figure out how they can make the Music: Not Impossible wearable an intrinsic part of the concert experience. This includes finding the best process for charging the devices, storing them, transporting them and putting them on people at the show.
There are so many different ways that we envision the technology being utilized, applied and incorporated into different experiences.”
They’ve also launched the Creator Network, where they made the software available to artists to figure out how to incorporate the tool into other forms of art. Through the network, artists across multiple disciplines are given the tools to alter the vibration tones, LED colors and more for projects. Some are currently working to incorporate the haptic technology into art installations and movies.
In December, the team plans to host a Creator Network workshop in New York. They’re also planning to unveil an art show that incorporates its wearable device at the Museum of the Moving Image in Queens, New York.
“There are so many different ways that we envision the technology being utilized, applied and incorporated into different experiences,” Onstott said. “We may not be able to do them all at once, but we want to make sure that as we scale one element, we're mindful of how that can kind of support support the next.”
