US announces deals with tech firms for national security review of AI models before release

The US government has struck deals with Google DeepMind, Microsoft and xAI to review early versions of their new AI models before they are released to the public.

The Center for AI Standards and Innovation (CAISI), part of the US Department of Commerce, announced the agreements on Tuesday, saying the review process would be key to understanding the capabilities of new and powerful AI models as well as to protecting US national security. These collaborations will help the federal government “scale (its) work in the public interest at a critical moment”, the agency said in a press release.

“Independent, rigorous measurement science is essential to understanding frontier AI and its national security implications,” said Chris Fall, CAISI director.

CAISI is an agency meant to facilitate collaboration between the tech industry and the federal government in developing standards and assessing risks for commercial AI systems. The agreement between the agency and the AI firms is focused largely on identifying national security risks tied to cybersecurity, biosecurity and chemical weapons.

OpenAI and Anthropic inked similar deals with the Biden administration two years ago and CAISI notes the agency has already completed more than 40 such evaluations, including on unreleased models. It is common for developers to share unreleased AI models with the government that have reduced or removed safety guardrails, CAISI said in its press release. This helps the government “thoroughly evaluate national security-related capabilities and risks”, the agency noted.

The new agreements come as fears grow that the newest and most powerful AI models – such as Anthropic’s Mythos – could be dangerous to release to the public; AI safety experts, government officials and tech companies fear the expansive capabilities of these models could help hackers exploit cybersecurity vulnerabilities at an unprecedented scale. Anthropic limited its rollout of Mythos to a few companies, and initiated the collaborative Project Glasswing to bring together tech companies “to secure the world’s most critical software”.

The New York Times and Wall Street Journal reported Monday the Trump administration was mulling over a potential executive order to create a government oversight process for these AI tools; the Administration has characterized this reporting as “speculation”.

Google andxAI did not immediately respond to a request for comment.

Microsoft announced a similar agreement in the UK on Tuesday with the government-backed AI Security Institute, which also focuses on safe AI development.

“While Microsoft regularly undertakes many types of AI testing on its own, testing for national security and large-scale public safety risks necessarily must be a collaborative endeavor with governments,” Microsoft wrote in a blog post about the two deals.