A smartphone camera can reveal more than what appears in its frames. Researchers report that ambient sound can make camera components vibrate, leaving subtle patterns in video that AI can analyze—even when the phone’s microphone is turned off.
How sound can leave a trace in video
The method relies on interactions among a phone’s camera hardware. The study describes smartphones with CMOS sensors, optical image stabilization (OIS) and autofocus. Ambient noise can vibrate the phone casing and moving camera lenses, creating tiny lens movements.
Those movements affect how the camera records an image. The researchers say the vibrations are amplified and encoded as rolling shutter artifacts: small distortions in pixels that are difficult to notice but can carry acoustic information.
AI methods can track motion between video frames and use those patterns to recover signals related to nearby sound. The researchers call this an “optical-acoustic side channel.” It does not record sound through the microphone; it draws information from changes in the camera’s images.
What the experiments could identify
In experiments involving ten smartphones, the team used video alone, with the microphone turned off. The phones were placed near speakers on a table, while the cameras pointed at either the table or the ceiling.
The researchers reported nearly 81 percent accuracy when classifying ten spoken numbers, 91 percent when distinguishing 20 different speakers, and 99.5 percent when classifying gender. These results show that camera artifacts can contain signals useful for classification under the tested conditions.
The findings do not mean that every phone camera can clearly capture conversations in every setting. The reported experiment involved specific phones and a setup with speakers nearby. Still, it demonstrates that video can carry information produced by sound, even when the sound source is not visible.
Privacy and possible uses
The technique could reveal something about a person in a room without showing them in the video. The study says gender could be determined with nearly 100 percent accuracy in its tests. That raises a privacy concern: camera access could potentially expose information about nearby voices beyond the visible scene.
Kevin Fu, a professor of electrical engineering and computer science at Northeastern University, also points to possible uses in law enforcement. He says the approach could offer evidence about whether someone was likely speaking in a room, including in legal cases or investigations. Such an inference would come from the analyzed video signals, not from an ordinary microphone recording.
Fu describes the effect as a very rudimentary microphone. His explanation is that image stabilization hardware can allow a lens to move slightly in response to nearby speech, modulating the voice onto the image and changing its pixels. The study notes that thousands of these movements could be recorded per second.
Hardware changes could reduce the signal
The researchers propose changes aimed at the physical causes of the side channel. Their suggestions include increasing shutter speed, randomizing shutter patterns and mechanically blocking lens movement.
They say combining defenses could reduce the method’s accuracy to a random level. The proposals focus on limiting the camera artifacts that make acoustic information recoverable in the first place.
The broader lesson is that a video stream may contain traces of sound even when a device is not using its microphone. For smartphone privacy, the study draws attention to the camera’s moving parts and image-capture process as another route through which nearby activity might become measurable.