A crowd’s roar can signal a baseball home run even when the play is out of sight. Paris Smaragdis wants computers to pick up on sounds like that and understand what they reveal about the world around them.
At MIT, his work explores machine learning for sound separation, devices that can augment human hearing, and tools for working with music. The goal is to help machines listen well enough to identify events and distinguish sounds that overlap.
From making sounds to understanding them
Smaragdis grew up in Greece and started out playing classical guitar. His interests shifted when his father brought home a synthesizer with a feature that let him create his own sounds. The possibilities drew him toward electronic music and the tools used to shape it.
He went to Berklee College of Music in Boston, where computers and synthesizers offered a way to create sounds that had not been heard before. He also gravitated toward jazz fusion and chose bass as his major, drawn to what he saw as the instrument’s less explored sonic territory.
At Berklee, he worked with mentor Richard Boulanger on tools for using computers to write, produce, and enhance music. A project with MIT Media Lab Professor Barry Vercoe led to graduate study in Vercoe’s group, where Smaragdis completed a master’s degree and PhD, then a postdoc.
That experience moved his attention from computer-generated music toward computers that could understand sound. While many Media Lab students used signal processing, he focused on machine learning as a way to teach computers to interpret audio.
Why listening is a machine-learning problem
Recognizing sound is more complicated than assigning a label to a recording. A machine may need to tell a cat’s meow from a violin, or separate several sources that are audible at once. Smaragdis argues that computers need to learn how to listen because people do not have a complete vocabulary for describing listening themselves.
His research group works on separating sound sources and on tools that help people understand, analyze, manipulate, and create audio. These problems connect basic questions about perception with practical uses: a system that can detect a meaningful sound might help identify an event even when a camera misses it.
One early example came from work at Mitsubishi Electric Research Labs. Smaragdis helped build AI systems that analyzed video soundtracks to infer what was happening. In a soccer recording, a surge in crowd noise could help flag a highlight, even when identifying one from images alone was difficult.
The system was later used in electronic devices, including a personal video recording system Mitsubishi was developing, as well as traffic monitoring systems and cell phones. The same general idea—treating sound as evidence about an event—also points toward uses such as locating a traffic accident from the sound of tires and metal, or detecting trouble in an industrial drill.
Audio tools for creators and archivists
After Mitsubishi, Smaragdis joined Adobe and created an audio research team. Its work focused on making tools that let artists, musicians, and producers manipulate sound without requiring extensive technical expertise.
He later joined the University of Illinois at Urbana-Champaign, where his group brought newer machine-learning methods into audio processing. One project used neural networks to separate individual sounds from mixed recordings. That work led to an unusual collaboration with the team behind the 2021 documentary mini-series “The Beatles: Get Back.”
The production team asked Smaragdis to help clean up archival Beatles recordings. The material presented varied challenges: one clip included John Lennon playing music while Paul McCartney read newspaper headlines and joked about George Harrison. The team sought a method flexible enough to handle different problems across recordings, rather than a separate fix for each one.
The work required a different measure of success from academic research. The result needed to sound right, even if it did not fit neatly into a formula. Smaragdis’s contributions helped the team win an Emmy Award for sound editing for a nonfiction or reality program, in 2022.
Bringing sound research to MIT
After 15 years at the University of Illinois, Smaragdis joined MIT in 2025. His appointments span music and theatre arts, electrical engineering and computer science, and the MIT Schwarzman College of Computing, reflecting the mix of fields involved in understanding sound.
That combination of music and computing has shaped his work from the start. Today, the questions extend from how machines separate sounds to how they can help people hear more of their surroundings. A machine that can listen effectively could make audio useful as a direct source of information, whether the task is monitoring equipment, finding an event, or working with a recording.