What Q-Star Could Mean for OpenAI’s Push Toward AGI

Reports described Q-Star as an OpenAI algorithm able to solve some grade-school math problems on its own, including problems absent from its training data. The claims, and the account of a warning to the board, were disputed; the reported capabilities raised questions about reasoning, scientific work and safety.

WTF Index TERMINATOR
◄ Terminator 3 Idiocracy 0 ►

Reports of autonomous problem-solving and potential scientific capability raised concerns about AI progress and safety, though the claims were disputed.

What Q-Star Could Mean for OpenAI’s Push Toward AGI

Reports about an OpenAI system called Q-Star brought two questions into focus: whether AI could make progress on problems it had not seen in training, and how researchers should assess the risks of such progress. The claims were tied to reports of an internal warning to the company’s board, but accounts differed over whether that letter reached the board or influenced Sam Altman’s firing.

A reported step in mathematical reasoning

Q-Star, also written as Q* and pronounced Q-Star, was described as an algorithm able to solve grade-school math problems autonomously. According to the reporting, those problems were not included in its training data. That detail drew attention because it suggested the system might do more than reproduce answers encountered during training.

Some OpenAI researchers reportedly viewed the result as a possible step toward artificial general intelligence, or AGI. Their reasoning, as described in the reports, was that mathematical ability might signal more general, human-like reasoning. But a reported ability in one area does not by itself establish that a system has reached AGI.

The reports also described Q-Star as a possible route to scientific work. A system that can reason through problems could, in principle, help speed scientific breakthroughs. That possibility was presented alongside concerns about whether the necessary safety precautions had been taken.

What was reported about the warning

Reuters reported that researchers sent a letter to OpenAI’s board describing both the opportunities and potential dangers of the system. Reuters cited two sources familiar with the matter and said the letter was linked to Altman’s firing, though it was not the only reason. Reuters also said it had not seen the letter.

The account was contested. A source from The Verge said the board never received such a letter, meaning it could not have played a role in Altman’s firing. The Information reported on the Q* breakthrough described in the letter, rather than on the letter itself. These differences mean the reported warning and its role in the company’s leadership change remained uncertain.

The reporting also connected the work to an “AI Scientist Team,” formed by combining the former “Code Gen” and “Math Gen” teams. The group was said to be investigating how existing models could be adjusted to improve reasoning and eventually carry out scientific work. The reports named Jakub Pachocki and Szymon Sidor as researchers behind Q-Star, building on Ilya Sutskever’s work. A demo was reportedly circulating inside OpenAI for several weeks.

Why the AGI rumors gained attention

Q-Star became part of a wider set of comments and rumors about OpenAI’s progress. In mid-September, the leaker “Jimmy Apples” tweeted that “AGI has been achieved internally.” Altman repeated the statement on Reddit, then edited his message to say an AGI “will not be announced with a Reddit comment” and that he had picked up on the meme.

In early November, Altman said current language models still needed “another breakthrough” on the path to AGI, and that scaling systems alone would not be enough. He said a system capable of discovering new physical phenomena would be required. At the APEC Summit 2023 on November 16, shortly before his dismissal, he described having been present four times when OpenAI pushed “the veil of ignorance back and the frontier of discovery forward.” He said the most recent time was a few weeks earlier. The reports suggested this might align with the leaker’s post, but did not establish that connection.

What remains unresolved

The Information also reported that Sutskever began a project called GPT-Zero in 2021. The idea was to give a language model more time and compute to produce an answer and make new academic discoveries. “Jimmy Apples” later brought up the codename “Zero” in connection with the rumors around the letter.

Taken together, these accounts describe research aimed at improving AI reasoning, while leaving important questions unanswered. They do not establish that Q-Star achieved AGI, confirm every claim about its capabilities, or settle whether a board letter existed. The reported math results and safety concerns instead show why progress in reasoning can prompt both expectations about scientific uses and scrutiny of how a system is developed.