From The Verge · Antonio G. Di Benedetto · · 3m
Running AI on your own machine
A Verge reviewer tests local AI models to avoid cloud services, finding useful tasks but also frustration and limits.
In brief
A Verge reviewer tests local AI models to avoid cloud services, finding useful tasks but also frustration and limits. Antonio G. Di Benedetto tested local AI models on his Mac Studio to keep personal data private, completing tasks like organizing his Steam library and analyzing financial records, but found the…
Why local AI appeals to one reviewer
Antonio G. Di Benedetto has avoided cloud AI services because they require sharing personal data with OpenAI, Google, Microsoft, or Anthropic. Running models locally on his own computer appeals to him.
Testing on a powerful Mac Studio
Di Benedetto is testing local AI on an M five Ultra Mac Studio with two hundred fifty six gigabytes of unified memory, which can run almost any model available.He is also testing on smaller devices.
Choosing a model to start with
Di Benedetto installed Hermes Agent, a free open-source desktop app for macOS, Windows, and Linux.
The overwhelming choice of models
There are more models out there than Di Benedetto can count, each with specialized uses. Generally, models with many billions of parameters can do more than smaller ones with only a few billion.
First task: a daily briefing
Di Benedetto set up Hermes to scan his email and calendar each morning at seven thirty am, telling him anything worth immediate attention and giving a short weather report.
Organizing over four hundred Steam games
Di Benedetto asked Hermes to reorganize his Steam library. He has over four hundred games on Steam and wanted them categorized by genre while keeping specialized categories like favorites, co-op titles, and party games.
Giving Hermes access to complete the task
To reorganize his games, Di Benedetto had to register a Steam web API key and give it to Hermes.
Tasks he would not trust to the cloud
Di Benedetto had Hermes analyze his financial records and create a laptop comparison spreadsheet. He would not upload sensitive financial data to the cloud, and the laptop information was under embargo, so keeping it local…
Automating benchmark testing
Di Benedetto runs all laptop benchmarks manually, repeating each test three times to take the average.He is working with Hermes to automate this process by creating Python scripts, though it remains a work in progress.
The limits of local AI models
Even powerful local models do not make a magical assistant that can do anything. Di Benedetto's daily briefing has already broken multiple times, and things still do not always work as intended.
How Di Benedetto views the technology
Di Benedetto views Hermes and local AI as just a tool, a computer program to run on his computer.
What comes next for Di Benedetto
Di Benedetto hopes to remain vigilant and cautious while continuing to test and learn with local AI. He plans to find more useful tasks for Hermes to handle and develop the briefing feature over time.
The essence
Antonio G. Di Benedetto tested local AI models on his Mac Studio to keep personal data private, completing tasks like organizing his Steam library and analyzing financial records, but found the technology still unreliable and requiring careful setup.
