Google's AI cost-saving projects for Kubernetes

Google Cloud's cost-saving projects

Google Cloud donated three projects to the Cloud Native Computing Foundation to reduce compute costs for idle AI agents on Kubernetes.

“Act 2 is a few years back; Kubernetes became the de facto for inference and training”

AI agents idle 90-95% of time

AI agents sit idle 90% to 95% of the time waiting on models, test runs or human approvals.

Agent Substrate reduces costs 10x

Agent Substrate saves agent memory and files when suspended, restoring work with 10x cost reduction on Kubernetes.

“We've seen 10x reduction in cost in terms of running Agent Substrate on Kubernetes versus running these agents directly on dedicated virtual machines and containers”

gVisor blocks security risks

gVisor sits on host kernel to block system calls and audit agent activities.

“Agents and agent harnesses are designed to solve a problem, and to try to solve a problem, until they can’t solve the problem”

KubeRay for distributed training

Google Cloud works with Anyscale Inc. on KubeRay to run Ray platform natively on Kubernetes.

OpenRL for reinforcement learning

OpenRL teaches post-trained models guardrails using reinforcement learning on Kubernetes.

“In the past, to do reinforcement learning, you needed to hire a dozen research scientists with PhDs”