Noise-canceling headphones usually reduce unwanted sound. A prototype called semantic hearing explores a more selective approach: users could turn down some sounds while bringing others, such as alarms or birds, forward.
The system is still under development, but its design points toward earphones and hearing aids that respond to what is happening around the wearer, rather than treating all background sound alike.
How semantic hearing works
The prototype connects standard noise-canceling headphones to a smartphone app. The headphones’ microphones, already used to cancel noise, also capture sounds in the wearer’s surroundings. A neural network running on the phone identifies those sounds and adjusts their volume in real time according to the user’s preferences.
Researchers from the University of Washington developed the system and presented their work at the ACM Symposium on User Interface Software and Technology (UIST) last week. They trained the network using thousands of audio samples from online data sets, along with recordings gathered in noisy environments.
After training, the network could recognize 20 everyday sounds. The examples included a thunderstorm, a toilet flushing, and glass breaking. The goal is to let the wearer choose which kinds of sound to soften and which to hear more clearly.
Early tests show promise, with limits
Nine participants tried the prototype while walking through offices, parks, and streets. The researchers found that it could muffle or boost sounds even in situations that were not part of its training.
Speech separation was a weaker point. The system had some difficulty distinguishing human voices from background music, especially rap music. That matters because clearer speech is a central need in many listening situations, including noisy public places.
Marc Delcroix, a senior research scientist at NTT Communication Science Laboratories in Kyoto who studies speech enhancement and recognition, said the work was a meaningful step for the field. He described it as the first proposal of a complete real-time binaural target sound extraction system, while noting that related ideas have appeared in speech separation research.
Potential uses beyond everyday listening
One possible application is hearing support. Hearing aids can be of limited use in noisy environments, and semantic hearing suggests a way to give wearers more control over competing sounds. Shyam Gollakota, an assistant professor at the University of Washington who worked on the project, said the idea is to use machine-learning algorithms to extract sounds of interest from the environment.
Selective listening could also be useful at work. Health-care, military, and engineering professionals may need to concentrate on particular sounds. Factory and construction workers, meanwhile, might want to protect their hearing while remaining able to communicate.
These are possible directions rather than established uses: the system described remains a prototype, and its real-world performance will depend on how well it handles the mix of sounds people encounter.
More control can narrow what we hear
The ability to choose which sounds are present could offer practical benefits, but it also changes how people experience their surroundings. Mack Hagood, an associate professor of media and communication at Miami University in Ohio and author of Hush: Media and Sonic Self-Control, said sound filtering could help communication when people deliberately focus their attention.
There is a trade-off. When listeners decide in advance what they want to hear, they may miss an unexpected sound or an experience they would have enjoyed. Semantic hearing therefore raises a question alongside its technical promise: how much should people shape the soundscape around them?
For now, the prototype shows one way headphones might move beyond simply reducing noise. With sound recognition and user control, they could become tools for choosing which parts of the audible world deserve attention.