Cost-Per-Insight: The Metric AI Operations Is Missing

Every AI operations dashboard measures cost-per-token. Input tokens. Output tokens. Cached tokens. Token spend by model, by agent, by hour.

This is accounting. It tells you how much you spent. It tells you nothing about what you got.

The wrong metric

Cost-per-token measures volume of computation. But computation isn't the product. Insight is the product. A 50,000-token output that restates what you already know cost more than a 2,000-token output that surfaces the one blind spot that changes your decision — and delivered less.

When you optimize for cost-per-token, you optimize for efficiency of production. When you optimize for cost-per-insight, you optimize for efficiency of learning.

These are not the same thing.

Defining cost-per-insight

Cost-per-insight is the cost to reach the actionable delta that changes a decision.

Not the cost to produce output. Not the cost to generate text. The cost to arrive at something you didn't know, that matters, that changes what you do next.

In one regulatory mapping task, a local model surfaced 80% of the compliance obligations correctly. The cloud review identified a single cross-article dependency that changed the entire compliance strategy. That dependency — not the 80% — was the insight. One sentence. $0.02. Decision changed.

This reframes every AI infrastructure decision:

• A local model that produces a reviewable 5/10 draft at near-zero cost, elevated to 7/10 by a cloud-tier review, has a lower cost-per-insight than a cloud model producing an 8/10 from scratch — because the cloud budget was spent entirely on the delta, not on structure.
• A task that runs through three inference tiers looks expensive on a token dashboard. On a cost-per-insight basis, it may be the cheapest path to the decision.
• A large model call that produces 10,000 well-formatted tokens of information you already had? Infinite cost-per-insight. It cost real money and produced zero delta.

The junior analyst model

Every experienced operator knows this pattern: a junior analyst who produces a structured, reviewable draft isn't measured by the quality of the draft. They're measured by how much senior time they save.

A 5/10 draft that gives the senior analyst something to react to, disagree with, and refine is worth more than no draft at all — even if the senior could have written an 8/10 from scratch. Because the senior's time is the expensive resource. The draft is infrastructure.

Local inference is this junior analyst. It works for near-zero marginal cost. It never sleeps. It produces structured, on-topic, reviewable output. It handles the 60% of the work that is high-volume, low-leverage: enumeration, formatting, first-pass analysis, template adherence.

The senior — whether that's a cloud model, a human expert, or both — focuses on the 40% that is judgment: synthesis, cross-referencing, strategic framing, the insight that changes the decision.

Measured, not theoretical

We ran three tasks of increasing complexity through a local 70B parameter model, then reviewed each output with a cloud-tier model. Threat modeling. Regulatory mapping. Competitive intelligence.

The insight density didn't decrease. In two of three cases, the reviewed output was better than what the cloud model would have produced alone — because the local draft gave the reviewer something to push against, not a blank page.

Why this matters at scale

A single team running a few AI tasks won't notice the difference. The economics change when you have ten agents running heavy workloads:

That's a 70–90% reduction in AI operations cost. Not by using worse models. By refusing to spend executive-tier compute on junior-analyst work.

The routing principle

The pattern is simple:

This isn't about choosing between local and cloud. It's about spending cloud tokens on insight, not formatting.

The structure was free. The first-pass analysis was free. The cloud budget was spent on the part that mattered: judgment.

A metric, not a product

Cost-per-insight isn't a feature you buy. It's a lens for evaluating every AI infrastructure decision you make. It applies whether you're running local models, cloud APIs, or both. And once you start measuring it, you can't unsee it.

The questions it forces:

• What did this compute actually produce that we didn't already know?
• Was the cloud budget spent on judgment — or on formatting?
• What's the cheapest path to the insight that changes the next decision?

This is not a local-versus-cloud argument. It's a resource allocation argument. Cloud inference is essential — but it should be spent on the delta that matters, not on structure a cheaper tier could have produced.

If you can't answer these questions, you're optimizing tokens. Not insight.

The metric is cost-per-insight. Everything else is accounting.