AI vendors can't be trusted to secure their systems. Newsrooms need to act accordingly
There were two big lapses in the Hugging Face / OpenAI hacking story: the lack of security protections on OpenAI's end, and Hugging Face's vulnerable production database. That should worry anyone who uses AI with sensitive data.
The biggest technology story this week was how a combination of OpenAI models hacked into third-party AI provider Hugging Face and breached its production database. The incident was initially spun as a sort of partnership between the two companies, but it seems like that’s not what went down at all.
“OpenAI failed to properly configure what it called a ‘highly isolated environment,’ allowing a testing sandbox that should have been completely secluded from the internet to actually connect to the internet.”
That’s actually one of at least two lapses here: not only did OpenAI fail to properly isolate its models, but Hugging Face’s production database was in a state where those models could hack into it. The whole thing does not speak well of security practices at AI vendors overall.
We’re being asked to share more and more private information with model vendors. The standard protection they offer — at least, to their paying customers — is that your data will not be used for model training purposes. That’s all well and good, but your data is still hitting their servers, potentially being logged for an extended period in such a way that it could, in theory, be accessed by their employees. Even before we bring in the possibility of hackers, that leaves your private data open to being accessed via subpoena, an unscrupulous employee, or, indeed, an unscrupulous vendor. (Consider that Uber breached at least one journalist’s privacy and considered hiring an opposition research firm. Do we really think AI vendors are more ethical? Why?)
Leaving a production database in a state where it could be breached is the icing on the cake. In this case, the models weren’t even harnessed to hack Hugging Face — they did so autonomously to cheat a test. Imagine what they might do if they were intentionally pointed that way. Hacking is becoming cheaper and easier: once the preserve of talented technologists, this story proves that the latest frontier models can effectively find and exploit vulnerabilities in systems. And apparently, AI vendors can’t be trusted to secure their own infrastructure. The combination of us being encouraged to share more and more data, the inherent risks of centralizing that data, the dubious security of the places we’re being asked to share it, and the obvious shadiness of some of the companies involved should give us all pause.
For newsrooms and anyone dealing with sensitive or private data, particularly relating to source materials or journalism in progress, this weak security environment is not enough. We need zero data retention contracts at minimum, but the only real way to be sure nobody can access our information is to use confidential computing environments and, ultimately, local models. Anything less leaves our work open to cowboys, hackers, and, apparently, misconfigured robots.
