The National Institute of Standards and Technology is moving generative AI evaluation into a more structured phase. Its new NIST GenAI program is designed to measure what generative AI systems can do, where they fall short and how content authenticity tools might help address misleading AI-generated material.
What NIST GenAI is designed to assess
NIST, a U.S. Commerce Department agency that develops and tests technology for the U.S. government, companies and the broader public, announced NIST GenAI as a program focused on generative AI technologies. The scope includes text-generating AI and image-generating AI.
The program will release benchmarks, support work on content authenticity detection systems and encourage software that can identify the source of fake or misleading AI-generated information. In plain terms, the effort is aimed at making generative AI easier to evaluate and making synthetic media easier to scrutinize.
NIST described the program as a series of challenge problems meant to evaluate and measure the capabilities and limitations of generative AI technologies. Those evaluations are intended to help identify strategies that promote information integrity and guide the safe and responsible use of digital content.
The first pilot starts with AI-written summaries
The first NIST GenAI project is a pilot study focused on distinguishing human-created media from AI-generated media. It starts with text, an area where detection tools have often been difficult to rely on.
NIST GenAI is asking teams from academia, industry and research labs to submit two kinds of systems. One group, called generators, will produce content. The other group, called discriminators, will try to identify AI-generated content.
For this pilot, generators must create 250-words-or-fewer summaries from a supplied topic and a set of documents. Discriminators must then determine whether a given summary is potentially AI-written.
NIST GenAI will provide the data needed to test the generators. The program says systems trained on publicly available data and systems that do not comply with applicable laws and regulations will not be accepted.
Key dates for the study
The pilot has a defined schedule. Registration is set to begin May 1. The first round of two scheduled rounds will close August 2.
Final results from the study are expected to be published in February 2025. That timeline gives participants a clear window to register, submit systems and take part in the evaluation process.
The schedule also shows that NIST GenAI is not only a broad policy response. It is beginning with a practical test: can systems generate short summaries under controlled conditions, and can other systems reliably flag whether those summaries may have been written by AI?
Why the focus on deepfakes matters
The launch arrives as AI-generated misinformation and disinformation are becoming a larger concern. According to data from Clarity, a deepfake detection firm, 900% more deepfakes have been created and published this year compared to the same time frame last year.
Public concern is also high. A recent YouGov poll found that 85% of Americans were concerned about misleading deepfakes spreading online.
That context helps explain why NIST GenAI is emphasizing content authenticity and detection. Text, images and other digital content can be generated quickly, but identifying the source of that content is harder. NIST GenAI’s early work focuses on measuring those detection capabilities rather than assuming they already work well.
The source article notes that many services claim to detect deepfakes, but studies and testing have shown them to be shaky at best, particularly with text. That makes the pilot’s text-first approach important: it puts a difficult detection problem into a formal evaluation setting.
How this connects to wider U.S. AI work
NIST GenAI is part of NIST’s response to President Joe Biden’s executive order on AI. That order required greater transparency from AI companies about how their models work and established new standards, including for labeling content generated by AI.
The launch is also the first AI-related announcement from NIST after the appointment of Paul Christiano, a former OpenAI researcher, to the agency’s AI Safety Institute. Christiano was described as a controversial choice because of his doomerist views.
He once predicted that “there’s a 50% chance AI development could end in [humanity’s destruction].” Critics, reportedly including scientists within NIST, fear that Cristiano may encourage the AI Safety Institute to focus on fantasy scenarios rather than realistic, more immediate risks from AI.
NIST says NIST GenAI will inform the AI Safety Institute’s work. That link matters because the program is focused on measurable evaluations: benchmarks, challenge problems and pilot studies that can be used to understand real capabilities and limitations.
For now, NIST GenAI begins with a narrow but consequential question. If a short summary is placed in front of a detector, can that detector tell whether it may have been written by AI? The answer will help shape how future content authenticity systems are tested, compared and improved.