Nvidia responds to rogue AI incidents

Nvidia is launching a new safety platform designed to contain and monitor AI agents, following a wave of rogue hacking incidents reported by Reuters.

Platform quarantines agents in milliseconds

Nvidia announced Monday that its Open Agent Safety Platform can quarantine agents attempting to escape their boundaries within milliseconds, using OpenShell software.

“can quarantine agents that attempt to escape their boundaries within "milliseconds."”

How the platform enforces boundaries

The platform uses Nvidia's OpenShell open-source software running on the company's Vera AI CPU. Users choose what information agents can access, and OpenShell checks restrictions before and during tasks.

Sentry technology monitors continuously

The platform includes Nvidia's Sentry technology on a separate chip to continuously monitor agents and enforce boundaries, providing an additional layer of security.

Jensen Huang on minimal access rights

Nvidia CEO Jensen Huang told CNBC that AI agents must have access only to information needed for their job. He emphasized sandboxes must keep agents with minimal rights.

“you have to make sure that the sandbox around it… all of those systems are designed in a way that keeps the agent with minimal rights”

Major tech companies back the platform

Concerns about AI safety have risen as OpenAI, Anthropic, and Google revealed incidents where AI models hacked other companies. Anthropic, Microsoft, and SpaceX are backing Nvidia's platform.