From Futuriom · · 2m
hosted.ai cuts GPU idle time with virtualization
hosted.ai virtualizes GPUs to boost utilization from 10% to 80% for service providers.
In brief
hosted.ai virtualizes GPUs to boost utilization from 10% to 80% for service providers. hosted.ai's software shares GPU capacity among tenants, reducing idle time from 80-90% to 10-15% for inference workloads. Originally reported by Futuriom.
gpu-as-service market problem
gpu-as-service cloud market evolves rapidly with many neoclouds inefficiently using their gpus.
“a customer who needs half a gpu now, three in a few hours and ten in the overnight batch has to buy ten gpus, because they must provision for the peak”
hosted.ai solves idle gpu problem
hosted.ai co-founder and ceo ditlev bredahl says customers buy too many gpus for peak usage, leaving 80-90% idle.
“the other 80 percent to 90 percent of what is deployed sits idle”
gpu virtualization solution
hosted.ai software virtualizes gpus so multiple tenants share capacity in oversubscribed way.
why this model matters
a gpu never has more than one customer unlike cpus, ram, storage, and bandwidth in other infrastructure.
vmware for generative ai
hosted.ai founders call their gpu virtualization approach vmware for generative ai.
founders' tech background
ceo bredahl and coo james withall built uk2, a european web hosting platform similar to godaddy.
onapp and previous ventures
bredahl and withall built onapp, a cloud management platform for cdns, telcos, and carriers to compete with aws.
onapp results
onapp gained over 3000 service provider customers and was sold in 2021.
gpu utilization improvements
a gpu billing 730 hours per month for single tenancy can bill 1825 hours per month on hosted.ai platform.
revenue impact
utilization moves from 10-15% typical of inference to 80-90% range, achieving 2.5x sellable hours per gpu.
mesh capacity sales
hosted.ai platform frees up 70% of existing gpu capacity for sale through the mesh.
mesh revenue targets
hosted.ai expects $250m to $350m in mesh revenue this year with over $500m in pipeline.
coldstart plan
coldstart helps neoclouds start up in locations with vacant land, power, and shell.
coldstart process
hosted.ai finds anchor tenants, sources hardware with nvidia, and manages end customers for coldstart.
future outlook
hosted.ai says demand for gpu virtualization has barely started with bright future ahead.
The short version
hosted.ai's software shares GPU capacity among tenants, reducing idle time from 80-90% to 10-15% for inference workloads.
