nightbench

note · 22 October 2026

October 2026: what actually runs our office at home

We run an office of AI agents. This is one week of its usage log, read straight from the database: which models did the calls, how many were local, what it cost, and what came out the other end. No benchmark scores here, only our own usage.

105,689model calls
28%served locally
$163hosted spend
528tasks marked done

What we measured

The window is 15 October 00:00 UTC to 22 October 02:43 UTC: seven full calendar days plus the first hours of the 22nd (marked with * below). Source: the table where the office logs every model call (provider, model, tokens, cost). "Local" means our own Ollama servers, logged at $0. "Hosted" means everything else: OpenRouter models, including the free-tier ones, and a small amount of Anthropic.

The setup

Calls, tokens and cost by model

providermodelcallsinput tokoutput tokcost
localornith-1.5:35b18,552315.1M6.6M$0.00
localornith-1.5:9b3,23211.8M1.5M$0.00
localornith:9b7,58927.6M4.4M$0.00
openrouterdeepseek/deepseek-v4.1-flash73,8231,682.5M107.7M$161.55
openrouterminimax/minimax-m3.1-flash:free78917.9M1.8M$0.00
openrouternvidia/nemotron-3-super-120b-a12b:free69317.4M0.4M$0.00
openrouternvidia/nemotron-3-ultra-550b-a55b:free3025.3M0.1M$0.00
anthropicclaude-haiku-5-52841.7M0.2M$0.36
openrouterqwen/qwen3.7-flash2194.3M0.1M$0.11
openrouterz-ai/glm-5.31253.0M0.2M$1.13
openrouterinclusionai/ling-3.1-flash811.8M0.3M$0.16
all models105,6892,088.5M123.4M$163.31

M = million tokens. The log has a third local tag, ornith:9b, next to ornith-1.5:9b; we list it as logged. Free-tier hosted models show $0 but are still hosted, so not "home".

Local versus hosted

By calls, 29,373 of 105,689 (27.8%) were served locally and 76,316 (72.2%) by hosted models. By tokens local models handled 17% of input and 10% of output. One hosted model, deepseek/deepseek-v4.1-flash, made 70% of all calls and $161.55 of the $163.31 spend. Its calls are large: about 23k input tokens on average, against about 17k for the local 35b.

So "mostly local" is not true of our week: about a quarter of calls ran at home, and the money went almost entirely to one hosted model. Local was ahead on only one day, 20 October.

Per day

Calls per day, local versus hostedOct 15: 4634 local, 15060 hosted; Oct 16: 2460 local, 7529 hosted; Oct 17: 4213 local, 14385 hosted; Oct 18: 2878 local, 6289 hosted; Oct 19: 4216 local, 13758 hosted; Oct 20: 6842 local, 4221 hosted; Oct 21: 3532 local, 12724 hosted; Oct 22*: 598 local, 2351 hosted0k5k10k15k20kOct 15 local 4634Oct 15 hosted 15060Oct 15Oct 16 local 2460Oct 16 hosted 7529Oct 16Oct 17 local 4213Oct 17 hosted 14385Oct 17Oct 18 local 2878Oct 18 hosted 6289Oct 18Oct 19 local 4216Oct 19 hosted 13758Oct 19Oct 20 local 6842Oct 20 hosted 4221Oct 20Oct 21 local 3532Oct 21 hosted 12724Oct 21Oct 22* local 598Oct 22* hosted 2351Oct 22*

local hosted · calls per day

day (UTC)localhostedtotallocal share
Oct 154,63415,06019,69424%
Oct 162,4607,5299,98925%
Oct 174,21314,38518,59823%
Oct 182,8786,2899,16731%
Oct 194,21613,75817,97423%
Oct 206,8424,22111,06362%
Oct 213,53212,72416,25622%
Oct 22*5982,3512,94920%

* 22 October covers only 00:00 to 02:43 UTC.

What came out

In the same window the office marked 528 tasks as done (status done, last updated inside the window). That is about 5.0 done tasks per 1,000 model calls, or roughly $0.31 of hosted spend per finished task. Caveats: a task can take many calls, some calls are retries or planning that never end in a task, and "done" is the office's own label, not an independent quality check.

What we take from it

These numbers describe our usage pattern only: one week, one office. They say nothing about how any model would do for you. For tested results see the nightly leaderboard and the cost page.

// off the bench

What these models build: Meet & Mint

Events and people near you on one map, a chat for every event, and you can join alone. The local models ranked above translate, sort and check the events in it every night. Many of the come-alone events on our sites are in the app too.

  1. 01Read tonight's results, pick a model that fits your card with Will it run?
  2. 02Take the raw numbers: data.json is the same file this page reads.
  3. 03See the models at work: open Meet & Mint, find an event near you, see who is going.
  4. 04Join alone, agree where to meet in the event chat, show up.
Meet & Mint app iconMeet & Mint: Events & Friends4.5 on the App Store · free

meetmint.app · privacy by default: approximate location only, who's going hidden until you join.