Large language models have changed what people expect from computers. Systems such as Claude, DeepSeek, and OpenAI’s GPT models can carry on conversations that feel natural, flexible, and fluent. Yet the comparison with children reveals a central weakness: machines still need far more language exposure than a child receives while learning a first language.
The data efficiency gap
People have been talking to each other for at least 100,000 years, and for almost all of that time, human children were the only learners known to reach perfect fluency in human language. Modern AI has added a second kind of fluent language user, but it reached that point by taking a very different route.
The core issue is called the data efficiency gap. A child can begin to grasp language by the first birthday and later produce grammatically correct sentences after hearing something like 10 million words, or 30 million on the high end. By contrast, a large language model can process a hundred thousand times more words than a person experiences while mastering a mother tongue.
Michael C. Frank, a cognitive scientist at Stanford University, captures the contrast bluntly: “The progress recently has been amazing,” he says, “But we still have to burn down a forest and scrape the entire sum of all human knowledge to re-create this milestone that happens in our living rooms over the course of a year.”
The scale is stark. Meta’s open-weight LLM Llama 3.1, released two years ago, used 15 trillion tokens in pretraining. Ethan Gotlieb Wilcox, a cognitive scientist and linguist at Georgetown University, says frontier models could be pretraining on 10 times more data. But the source material is not unlimited, and the easily available data supply could run dry perhaps as early as the 2030s.
What children show AI researchers
Children prove that language can be learned with much less input. A preteen raised in a linguistically rich home may have heard something in the vicinity of 100 million words. With literacy included, that number may reach maybe 300 million words by age 20.
Those numbers remain tiny beside the training diets of modern models. Wilcox says Claude has seen the amount of language an entire city will experience in one generation. If all the words used to train a modern LLM were printed on paper, the stack could reach past the International Space Station. The 100 million words heard by a human preteen would stack up just 20 meters.
That gap matters because it points to a practical research target. If scientists can understand how children learn so much from so little, AI systems might become more efficient. The source article points to possible uses including training AI effectively on video and creating chatbots for minority language communities.
The question also matters for cognitive science. Researchers want to know whether children need an inborn language instinct, whether language can be learned from experience, and whether some features of language learning reflect general constraints rather than biology alone.
The old debate meets modern models
The mystery is not new. In the 1950s, MIT linguist Noam Chomsky argued that children must be born with hardwired knowledge of grammar. He was responding to psychologist B.F. Skinner, who treated language acquisition as a product of conditioning and reinforcement.
Chomsky’s argument rested on the “poverty of the stimulus,” the idea that children hear too little and too imperfect a sample of language to infer grammar from experience alone. Richard Futrell, a linguist and cognitive scientist at the University of California, Irvine, summarizes the view this way: “His signature argument was, essentially, that language cannot be learned on the basis purely of statistics.”
That view shaped linguistics in the US for decades and influenced early computer science. During the 1950s and ’60s, researchers often tried to make computers understand language by explicitly coding rules. This symbolic AI approach lasted for decades, but it did not produce systems that could handle human language at scale.
Neural networks later returned to prominence. In the 2010s, cheaper hardware and a growing internet helped them improve. By 2018 and 2019, BERT and GPT-2 showed researchers that transformer models trained on billions of tokens could learn language patterns powerfully. In 2022, ChatGPT made that shift visible to a much wider public.
Large language models are not brains. They are statistical learners. Still, they challenged the assumption that grammar could not emerge from patterns in data. Alison Gopnik, a developmental psychologist at the University of California, Berkeley, says the surprising part is simple: “These things learn syntax.”
Testing baby-scale AI
The next question is whether machines can learn from child-size amounts of language. That is where BabyLM enters the story. Alex Warstadt, a linguist and data scientist at the University of California, San Diego, helped organize the annual competition after a 2022 exchange with AI researcher Leshem Choshen.
BabyLM asks researchers to train models on small data sets. Its main track uses a “developmentally plausible” corpus of just 100 million words, while the toddler-scale track uses 10 million. The data comes from storybooks, dialogue, movie subtitles, Simple English Wikipedia, normal Wikipedia, and transcripts of speech directed at children.
The models are tested on grammar benchmarks similar to those used in psycholinguistics. One kind of test compares sentences such as “The keys to the cabinet are on the table” and “The keys to the cabinet is on the table.” Humans show surprise when grammar breaks; models are evaluated using surprisal, a measure of how unlikely they expect a sentence or part of a sentence to be.
BabyLM has already complicated assumptions. Curriculum learning, which moves from simpler to more complex training data, was popular in the first round. But Aaron Mueller, a computer scientist at Boston University and one of the organizers, says transformers do not seem to need their data ordered that way to learn effectively.
The strongest BabyLM systems are not necessarily baby-like. The 2024 champion, GPT-BERT, combines next-token prediction with a BERT-like method that fills blanks in token sequences. When pretrained on about 100 million words, it beat Meta’s Llama 2 70B, pretrained on roughly 15,000 times that amount, on one BabyLM benchmark.
Why experience may matter
Even so, BabyLM models are not equivalent to modern commercial LLMs, and many cannot produce text. More important, their learning is unlike a child’s. Children do not learn from written text alone. They hear, see, touch, move, and interact.
That is why some researchers are turning to children’s own sensory experience. About 15 years ago, Frank’s Stanford lab began studying what babies actually see through headcams. The early findings suggested that children’s visual world is highly focused, shaped by short arms, nearby objects, and adult legs.
Frank and four colleagues later created SAYCam, a project that recorded two hours a week of each child’s life from six months to two and a half years of age. The project involved three babies, and the families allowed the video to be released for research.
Brenden Lake, a cognitive scientist and AI researcher at Princeton, used that kind of material with colleagues. In 2024, while working out of New York University, he and his team presented a model trained on 61 hours of raw SAYCam data that learned to identify objects and associate them with words.
The lesson is not that AI has already solved child language learning. It has not. The lesson is that children remain the benchmark for learning language efficiently, and their success may help researchers build smaller, more capable, and more grounded machines.