A Look at the University of Utah’s Accessibility Research Using Meta Neural Band

Meta Neural Band lets people control their Meta Ray-Ban Display glasses using simple and intuitive hand gestures — and it may open up new possibilities for more inclusive human-computer interaction (HCI) for people with a wide range of neuromotor abilities.

We recently announced a new research collaboration with the University of Utah to evaluate consumer-grade wrist wearables for people with different levels of hand mobility, with the goal of making human-computer interactions more accessible. To learn more about this exciting work, possible future applications for surface EMG technology, and the potential to make HCI more accessible for everyone, we sat down with Jacob A. George, Director of the Utah NeuroRobotics Lab and the Solzbacher-Chen Endowed Professor in the John and Marcia Price College of Engineering’s Department of Electrical & Computer Engineering and the Spencer Fox Eccles School of Medicine’s Department of Physical Medicine and Rehabilitation at the University of Utah.

The principal investigator on the project, Dr. George was a winner of Reality Labs’ 2021 Engineering Approaches to Responsible Neural Interface Design request for proposals. Using funding from that RFP, his team’s previous research demonstrated how surface EMG signals at the wrist remain viable for control, even in cases where the signal-to-noise ratio is reduced, for example following a stroke. Notably, research participants who were unable to extend their physical fingers were able to move all of the fingers of a virtual hand avatar using EMG. This earlier research showed the expressive potential of surface EMG in virtual environments beyond the physical capabilities of an individual.

Through this new collaboration, Dr. George’s team will work with Meta researchers and end users to examine how Meta Neural Band and Meta’s internally developed research algorithms can enable controls and serve as an interface for people with different levels of hand mobility. The team will use the wristband to measure electrical signals from muscles at the wrist and translating them into digital signals to control computers, smart home devices, and possibly even recreational mobility devices like the University of Utah’s TetraSki.

What Is Surface Electromyography (EMG)?

Surface EMG uses external sensors around the wrist to detect electrical muscle signals that control the wrist and hand — and it’s a technology that opens up an easy-to-use, convenient, and rich new form of HCI.

Surface EMG input is inherently inclusive compared to traditional physical controllers, enabling functional control for people with atypical anatomy as well as those with limited range of motion. Unlike camera-based systems that detect physical hand movements from the hands and fingers — or handheld joystick- and button-based controllers that need to be pressed and pushed — muscle signals at the wrist have the potential to provide control signals even if you can’t produce large movements or if you have fewer than five fingers on your hand.

Meta Neural Band is our first consumer wristband to deliver surface EMG-based HCI, specifically for Meta Ray-Ban Display glasses. And the potential for future applications is coming into focus.

The Intersection of Technology & Rehabilitation

Dr. George highlights the critical role that assistive and rehabilitative technologies play for people with conditions like muscular dystrophy, stroke, spinal cord injury, ALS, and/or limb loss. His lab focuses on developing accessible tools to help people regain functionality in their daily lives. For example, a person with a hand amputation may directly map their muscle signals to computer controls or learn to cook using a prosthetic hand. The goal is to create efficient interfaces that let people interact with their environment more dexterously.

Neural Interfaces & the Importance of Intention

In order to work, Meta Neural Band requires intentionality — people must focus on taking a specific physical action to control the technology effectively. By tapping into the control signals from muscles, which are more deterministic than the central nervous system, the wristband can accurately interpret motor signals and translate them into actions for various devices.

Assessing Accessibility

For people with muscular dystrophy, ALS, and other conditions, everyday activities like raising window blinds can present a challenge. Motorized smart home devices can help do the work, but their interfaces must be accessible for people with a range of abilities. What might be intuitive to some people — like turning their hand to control a smart thermostat — might be impossible for others.

The research project aims to assess how effectively individuals with neuromuscular conditions can use the Meta Neural Band for everyday tasks, ensuring that the technology can be customized to fit their unique needs. University of Utah researchers will work closely with the internal Meta research team to design experiments that evaluate and iterate on customized gesture controls based on ongoing user feedback. This co-design between the end users, engineers, and scientists is essential for creating technology that meets diverse needs.

EMG for All

Dr. George highlights the potential of surface EMG to enhance quality of life and increase independence for people with neuromuscular impairments. By focusing on accessible design and user collaboration, these technologies can empower individuals and pave the way for a more inclusive future.