AI labs sound the alarm on safety

Top artificial intelligence labs are publicly calling for slower development and stronger safety controls as AI agents increasingly find ways to exploit system vulnerabilities and evade human oversight.

CIOs caught between adoption and risk

Chief information officers are rapidly integrating AI agents into operations while watching risky examples emerge from labs, forcing them to balance innovation with security concerns.

Enterprises must enforce guardrails now

PwC's global chief AI officer says C-suite technology and security executives must collaborate to enforce proper guardrails, monitor AI agents, and track all tasks these systems perform across departments.

“This is a risk that enterprises need to be focused on, understand, and start planning for.”

Legal accountability falls on leadership

Department heads across supply chain, customer service, marketing, legal, and HR must track their digital employees. Claiming an agent made you do something won't hold up legally or morally.

“'The agent made me do it' is not going to be a defense from a moral or legal perspective.”

Cisco centralizes all AI agents

Cisco built MyAgent in August to centralize all company-authorized large language models, agents, and enterprise data into one platform, rejecting third-party AI assistants to prevent sprawl.

“We are going to cannibalize and kill every other AI assistant within the company.”

Cisco achieves rapid adoption at scale

Around 90,000 Cisco employees have access to MyAgent, with 50% daily adoption within two weeks. Employees can create their own agents, but around 700 have been approved by a centralized team.

Intuit bakes security into AI from day one

Intuit's GenOS operating system includes GenSRF, which tracks every AI request and records all responses. The company learned that retrofitting security after deployment is far harder than building it in initially.

“You don't want to try to retrofit the ability to enforce security and responsible AI foundations after the fact.”

Rogue agents mostly caught in testing

Intuit's CTO notes that disclosed AI agent incidents have mostly occurred during testing phases, and that Anthropic and OpenAI have shown willingness to slow development when issues arise.

Can't rely on AI companies alone

Intuit's CTO warns that expecting frontier AI companies to intrinsically do the right thing is unrealistic. Enterprises must build their own safeguards rather than trusting model developers.

“if you're going to rely on the model intrinsically to do the right thing, I think you're expecting too much of these frontier LLM companies.”

Secure acceleration, not slowdown

Lumen Technologies' chief technology officer argues that a broad slowdown isn't enforceable or automatically safer. Enterprises should pursue secure acceleration instead, since competitors and adversaries won't pause.

“The bad guys aren't going to slow down, other nations aren't going to slow down.”

Workday creates agent system of record

Workday built an agent system of record to manage all non-human digital workforce identities, used both internally and sold to customers. Training engineers on agent risk profiles is critical.

Responsibility cascades through teams

Workday's CTO emphasizes that security and compliance awareness must reach all employees, especially as AI handles coding, deploying, and releasing software. Teams must own their agent's rules and guidelines.

“It's got to be delegated down to the team who's driving these agents, who's in charge of the engine, the context window, the rules, and the guidelines.”

ServiceNow launches AI Control Tower

ServiceNow's AI Control Tower platform manages, observes, secures, and governs AI agents. Used internally and sold to customers, it's one of the company's fastest-growing products because it gives C-level executives peace of mind.

AI is a new form of insider threat

Zscaler's chief information security officer says security teams spent careers managing human risk but only have years of experience with AI. AI is non-deterministic, takes initiative, and acts like a new insider threat.

“AI is non-deterministic, it can take initiative, and it is effectively a new form of insider.”

Industry forms Open Secure AI Alliance

Cisco, ServiceNow, Workday, and Zscaler joined Nvidia-led Open Secure AI Alliance with dozens of firms to share ideas on developing open-source tools with proper safeguards around software and AI agents.

AI psychology remains largely unknown

Zscaler's security chief warns that leaders don't yet understand the characteristic psychology of AI. While humans are motivated by known incentives, the motivations of silicon-based intelligence remain unclear.

“I don't think we have begun to understand the characteristic psychology of AI.”

AI labs divided on safety approach

OpenAI, Anthropic, and Microsoft call for safety standards and coordination. Meta and Nvidia oppose slowdowns. Trump calls AI safety concerns a hoax and says only a strong president is needed as guardrail.

Congress unlikely to act on AI regulation

Congress has failed to enact any AI safety, privacy, or security bills in recent years. No serious effort to address AI regulation is expected in Washington until after midterms in November.

Poor data foundations derail AI pilots

Seven in ten data leaders report that AI project failures stem from unaligned or poor data foundations. Over half spend significant staffing hours manually reviewing and correcting AI agent outputs before deployment.

Manual verification inflates AI costs

With 87% of leaders needing to re-verify agent context, manual review processes add expense and reduce efficiency gains. Rising AI adoption has dramatically increased token spending, catching CIO attention.