The National Institute of Science and Technology (NIST) is launching a program that will assess quickly proliferating generative AI technologies that are being developed by research institutions – such as tools that generate text, images and videos.
The NIST GenAI program also is aimed at creating systems that will be able to identity whether text, images or videos are created by an AI model; a key step in pushing back against disinformation such as voice-cloning campaigns or deepfakes that can fool users into believing they are real.
The rollout of the NIST AI program was among several announcements this week by the U.S. Commerce Department, including draft publications from NIST addressing AI security and trustworthiness. They’re also among a flurry of moves by federal agencies, such as the Department of Homeland Security and CISA, that mark the six-months milestone since President Biden issued his executive order address about the need for the safe and secure development of AI technologies.
Testing and Evaluating
According to the release from the Commerce Department – of which NIST is a part of – the program is similar to others run by NIST, which tests and develops technologies for both the government and private sector. It will be an umbrella program that includes a platform for testing and evaluating generative AI technologies, with the evaluations playing a role in the work of the U.S. AI Safety Institute, which also is at NIST.
The evaluations will include evolving the creation of a benchmark dataset and running comparative analysis using metrics. In addition, according the NIST AI Program website, the initiative also will include “facilitating the development of content authenticity detection technologies for different modalities (text, audio, image, video, code) and “promoting the development of technologies for identifying the source of fake or misleading information.”
In announcing the new program and draft publications, Commerce officials echoed what others in the federal government – as well as private sector – have said, which is that AI holds the promise of delivering significant benefits to both the business world and society at large, but that it carries tremendous risk as well. President Biden’s executive order from October 2023 is taking a whole-of-government approach of ensuring that the benefits are realized while the risks are reduced.
“For all its potentially transformative benefits, generative AI also brings risks that are significantly different from those we see with traditional software,” Laurie Locascio, NIST director and under secretary of commerce for standards and technology, said in a statement. “These guidance documents will not only inform software creators about these unique risks, but also help them develop ways to mitigate the risks while supporting innovation.”
Pilot Study
NIST is kicking off a pilot study that will measure and understand how systems behave to better discriminate between content that is synthetic and content that is created by humans in both text-to-text and text-to-image situations.
“This pilot addresses the research question of how human content differs from synthetic content, and how the evaluation findings can guide users in differentiating between the two,” the agency notes. “The generator task creates high-quality outputs while the discriminator task detects if a target output was generated by AI models or humans.”
The test will include two teams: one of generators, who will be tested on their system’s ability to generate synthetic content created by large language models (LLMs) and generative AI tools, the other of discriminators, whose system will be tested for its ability to detect such synthetic content.
The More the Merrier
NIST is inviting teams from academia, other research labs and the tech industry to contribute to the pilot. The generator teams will create a summary of no more than 250 words of a topic and a set of documents. Discriminator teams will be given text summaries that may have been written by a human or AI and will have to detect which is true. The registration period for the pilot opens May 1 with the NIST-provided source data being released June 3.
The worry about AI-created disinformation is heightened by the high-profile elections happening this year in the United States, India and elsewhere. In its threat assessment this year, DHS expects bad actors to put a focus on elections, while cybersecurity firm Arctic Wolf found in a survey that disinformation was the top concern of election officials in the United States.
“For disinformation campaigns, AI algorithms can now be trained to analyze vast amounts of data, identify trends and mimic human behavior on social media platforms,” the survey authors wrote. “By deploying AI-driven bots or deepfake technologies, malicious actors can flood online spaces with misleading narratives, fabricated stories and manipulated media.”
