16.4 Million Tokens for Six Tenths of a Cent
Atlas processed 16.4 million tokens in the last 30 days. That was 2,448 requests, almost all of it on Qwen 3.6 27B running locally on a machine in my house.
Here is the number I keep coming back to. My total inference spend for the month was $0.0063. Six tenths of a cent.
That six tenths of a cent bought exactly 6 requests to GPT-4o, and those 6 requests existed for one reason: to prove that if my local model was ever down, Atlas could still fall back to a hosted API and keep running. The other 2,442 requests, and effectively all 16.4 million tokens, never left the building.
The same workload against a hosted frontier API would have run to dozens of dollars for the month, depending on the provider. Not a fortune. But the gap between "dozens of dollars, every month, forever" and "the electricity bill" is the whole argument, so it is worth sitting with.
The hardware is a rounding error
All of this runs on a single RTX 5060 Ti with 16GB of VRAM, in a box I built for around $1,000.
That is the part people expect to be the catch, and it just is not one anymore. A consumer GPU you can buy today holds a capable 27B model in memory and serves it fast enough to sit inside an interactive assistant. The thing that used to require a rack and a budget now requires one card and a weekend.
Today alone, Atlas ran a million tokens across 157 requests. That was not a launch day or a stress test. That was a Tuesday.
So what is it actually doing with all those tokens?
This is the question I get whenever I say the numbers out loud, usually with some skepticism, because a million tokens a day sounds like it must be doing something enormous.
It is the opposite. It is thousands of tiny jobs.
Atlas polls on a 30 minute cycle: world news, a set of topics I care about, specific stocks, my texts and WhatsApp messages, and a few other feeds. Each cycle, instead of asking the model to reason about everything at once, it fans the work out into many small, single-purpose calls. Things like:
- "Summarize this one article."
- "How does this new feature fit within the Atlas ecosystem?"
- "What about this message, if anything, is worth writing to long term memory?"
Each of those is a small prompt with a narrow job and a short answer. Individually they are nothing. There are just a lot of them, running around the clock, and they add up to 16.4 million tokens a month without any single request ever being large.
This shape matters more than it looks. Lots of small, well-scoped calls is exactly the workload where per-token API pricing quietly punishes you, because the cost scales with how much your assistant thinks, and a genuinely useful assistant thinks constantly in the background. It is also exactly the workload a local model handles happily, because once the hardware is paid for, the marginal cost of one more small job is zero. The economics of hosted inference and the economics of a good background agent point in opposite directions.
Local inference is a deployment strategy, not a hobby
I think local inference still gets filed under enthusiast tinkering, the AI equivalent of roasting your own coffee. That framing is out of date.
Run the model in your house and the data it reads stays in your house. My texts, my WhatsApp threads, the topics I follow, the notes Atlas writes to memory about my life: none of it is shipped to a third party to be logged, retained, or used to train the next model. That is not a privacy setting I toggled and have to trust. It is a property of where the computer is.
The rest follows from the same fact. No subscription to a company that can change its pricing next quarter. No rate limits deciding how much my own assistant is allowed to think today. No dependency on someone else's uptime, terms of service, or willingness to keep serving the model I built on. Nothing leaves without my say, because there is no outbound path unless I add one, the way I added those 6 fallback requests on purpose.
What is actually stopping you
Here is where I have landed after a month of watching the numbers.
The models are good enough. A 27B model running locally does the overwhelming majority of Atlas's work without ever reaching for a hosted API, and the 6 times it did were a deliberate safety net, not a rescue.
The hardware is cheap enough. One consumer GPU, roughly a thousand dollars, one time. Set against dozens of dollars a month forever, it pays for itself inside a couple of years and then keeps working.
So the constraint is not capability and it is not cost. It is the willingness to own the stack: to build the box, run the model, and take responsibility for your own inference instead of renting it by the token and handing your data over as part of the deal.
That is a real cost. It is your time and your attention, and for some workloads renting really is the right call. But it is worth naming honestly, because it is the actual thing in the way, not the price and not the models.
So the question I will leave you with is the one I had to answer for myself: what is actually stopping you from running your own inference?