Meta says it has developed AI models that can produce speech in more than 1,000 languages and recognize speech in more than 4,000. The company is making the models available on GitHub, with the aim of helping developers create speech tools for languages that current systems rarely support.
Why broad language coverage is difficult
There are around 7,000 languages in the world, but existing speech recognition models comprehensively cover only about 100. These systems often depend on large collections of labeled training data, such as audio paired with transcripts. Such data is available for only a relatively small group of languages, including English, Spanish, and Chinese.
The gap affects what developers can build. A messaging service that understands spoken language, or a virtual-reality system usable in different languages, depends on speech technology that can handle those languages. When training material is scarce, creating those tools becomes harder.
Meta’s researchers took a different route to reduce the need for transcripts. They retrained an existing model the company developed in 2020, which can learn patterns in speech from audio without requiring large amounts of labeled data.
Learning from recordings and text
The team used two data sets built around the New Testament. One included audio recordings and corresponding text gathered from the internet in 1,107 languages. The other contained unlabeled audio recordings in 3,809 languages.
Researchers processed the audio and text to improve their quality, then used an algorithm to match recordings with their accompanying text. They repeated the process with a second algorithm trained on the newly aligned material. According to the team, this approach helped the system learn languages more easily, including when accompanying text was unavailable.
Michael Auli, a research scientist at Meta who worked on the project, said the model’s learning could be used to build speech systems with very little data. He pointed to the uneven availability of good data sets: English has many, as do a few other languages, while languages spoken by, say, 1,000 people lack them.
What the models can do, and where they fall short
Meta’s researchers say the models can converse in over 1,000 languages and recognize more than 4,000. They compared their systems with models from rival companies, including OpenAI Whisper, and claim theirs had half the error rate despite covering 11 times more languages.
Those claims describe the researchers’ comparison; they do not mean the system is free from mistakes. The team warns that it may mistranscribe words or phrases. In some cases, a mistake could lead to an inaccurate or potentially offensive label.
The researchers also acknowledge a bias concern. Their speech recognition models yielded more biased words than other models, although the difference was only 0.7% more. That finding sits alongside the technical reach of the system as an issue developers may need to weigh when using it.
Questions about the training material
The project’s use of religious texts raises a separate concern. Chris Emezue, a researcher at Masakhane, an organization working on natural-language processing for African languages, said the Bible contains bias and misrepresentations. Emezue was not involved in the project.
Making models available through GitHub could give developers a starting point for speech applications in more languages. But wider language coverage does not, by itself, settle questions about accuracy, representation or the source material used in training. The researchers’ warnings and Emezue’s criticism point to considerations that remain relevant as these tools are developed and applied.