Artificial intelligence labs received the brunt of the criticism for their models escaping containment during safety testing, but it turns out they are not solely responsible.
Meta, Anthropic and OpenAI have had their AI models go rogue during testing. AI safety experts believe this shows that companies cannot control AI, while AI advocates think this has to do more with failing to set up proper testing environments set up by third-party testing companies.
Irregular — a third-party AI testing startup based in Israel — suffered a misconfiguration that allowed Meta’s and Anthropic’s testing environments to inadvertently access the internet.
“A misconfiguration by Irregular, an independent testing company Meta uses, inadvertently allowed one of our models access to the internet during evaluation. The model subsequently exploited a security vulnerability in a third-party service, in a manner similar to previously-reported instances with other companies,” a Meta spokesperson told the Daily Caller News Foundation about an incident that led the tech giant’s AI to hack another company’s during cybersecurity testing.
Irregular said that the cybersecurity breach is the “exact same evaluation-environment issue” that Anthropic disclosed that allowed its models access the internet and hack three separate organizations. An Irregular spokesperson said it resolved the issues and continues to work on a new standard to safely test AI models, per the Washington Post.
The Israeli startup serves as a niche but integral player in the artificial intelligence industry, backed with $80 million in venture capital funding from Sequoia and Redpoint Ventures, valued at $450 million, according to CNBC.
Irregular remains one of the few AI testing labs with the expertise to test frontier models, Sundeep Bhimireddy, the head of AI at the enterprise startup, Von, told CNBC.
“When they are testing these models, they don’t want to grade their own homework. They want independent testing that needs to be done by outside third-party vendors,” he explained, according to the outlet. Bhimireddy cited non-profit METR and public benefit corporation Apollo Research as the two other experts besides Irregular.
One AI expert believes the hacking incidents have more to do with proper testing setup than AI going rogue.
“Ultimately the news regarding Meta is part of a larger trend of these leading labs doing a poor job of setting up a proper testing environment. It is critical to understand that in all the incidents to date, none have involved an instance of ‘rogue AI.’ The efforts to paint it as such is nothing more than marketing theatre, something that several prominent voices in cybersecurity have already noted,” James Czerniawski, the head of emerging technology policy for the Consumer Choice Center, told the DCNF.
Irregular reportedly declined to say if any other of its clients were impacted by the same AI testing environment, according to The Record.
“This did not involve a sandbox escape or a sophisticated cyber action. There are no current open issues. Irregular is developing a white paper to share best practices for containment and securely running cyber evals,” an Irregular spokesperson told CNN.
One cybersecurity expert criticized the testing environment for the frontier models, arguing that evaluators should have more closely monitored the test and performed the tests on “air-gapped” systems not connected to an outside network.
“These incidents reveal how little care has been put into designing these [evaluations] and the security around them. They don’t monitor what’s happening. They’re just letting agents run wild both within their environment and in partner testing situations,” said Zack Korman, CEO and cofounder of Embroidery, an AI cybersecurity company, told the Post.
(DCNF)
