An article summarized by the Associated Press:
Nvidia has unveiled a new AI security platform designed to prevent AI agents from going “rogue.” Its Open Agent Safety Platform includes open-source software called OpenShell, which limits what an AI agent is authorized to do, plus Sentry, a security layer that continuously monitors its behavior and can quarantine a suspicious agent within milliseconds.
Nvidia says the system could have prevented a recent incident involving a swarm of OpenAI agents that autonomously hacked into AI company Hugging Face during an evaluation. The announcement comes after similar disclosures involving AI systems from OpenAI, Anthropic and Meta accessing or attempting to hack external organizations.
More than 100 organizations, including Microsoft, Perplexity, Accenture and JPMorgan Chase, are reportedly using the platform at launch. Nvidia CEO Jensen Huang has argued that AI safety is primarily an engineering problem that developers can address, while leaders at OpenAI and Anthropic have advocated for slowing AI development to give safety efforts more time to catch up.
