Appendix C — Appendix C: Cost and unit economics

Every applied chapter ended with a cost callout in unit-economics terms. This appendix gathers the cost discipline in one place: how to attribute cost, the levers that move it, and how to present it to someone who controls a budget.

C.1 Translate cost per token into cost per outcome

A price of “$0.40 per million output tokens” means nothing to a business sponsor. Translate it into the unit they care about: cost per ticket triaged, per document summarized, per question answered. That is the number that compares against the manual cost it replaces.

from dotenv import load_dotenv
load_dotenv()
from gaba.llm import cost_estimate, MODEL_DEFAULT

# A triage call: a short prompt in, a one-word category out.
per_call = cost_estimate(MODEL_DEFAULT, input_tokens=90, output_tokens=3)
print(f"cost per ticket:        ${per_call:.6f}")
print(f"cost for 10,000 tickets: ${per_call * 10_000:.2f}")

# Compare to the manual alternative.
manual_per_ticket = 0.50  # e.g., 30 seconds of an agent's time
print(f"manual cost for 10,000:  ${manual_per_ticket * 10_000:,.0f}")

cost per ticket:        $0.000010
cost for 10,000 tickets: $0.10
manual cost for 10,000:  $5,000

That contrast, a few dollars of model cost against thousands of dollars of human time, is the case for the project. It is also deliberately narrow: it counts only the inference, and a real proposal adds the build and maintenance cost, but the per-outcome framing is what makes the comparison legible.

C.2 The levers that move cost

In rough order of impact:

#	Lever	The move	Typical saving	Chapter
1	Context size	Pack less into each prompt: fewer RAG chunks, tighter instructions	Proportional: halving the context roughly halves input cost, which usually dominates the bill	Ch 10
2	Model choice and routing	Send the easy bulk to a cheap model; escalate only the hard cases	The price ratio between the models on whatever you downshift; the blended bill is the volume-weighted average of the two prices	Ch 2, 12
3	Prompt caching	Put the stable prefix (system prompt, instructions, examples) first	The provider’s cache discount, applied to the cached share of input tokens	Ch 2
4	Batching	Queue work that is not time-sensitive on the batch API	Often around 50 percent off, but check your provider’s current batch pricing	Ch 2
5	Self-hosting	Replace the API with a GPU you keep busy	Anything from negative to large; it pays only at high, steady volume past the break-even, usually when replacing an expensive model	Ch 4

Check the levers in order. The first is free and immediate, while the last is a project with an up-front cost and a utilization condition attached.

C.3 When to self-host or fine-tune

Both are investments that pay off only at scale, and both have a break-even you can compute before committing (Chapter 4 for self-hosting, Chapter 15 for fine-tuning). The shape is the same: a fixed up-front cost repaid by cheaper per-call cost, so the question is always “do we have the volume,” which we answer with the break-even arithmetic; a common mistake is to answer it by instinct.

C.4 A cost worksheet

For any feature, copy this table and fill in the right column, top to bottom; the last three rows are the ones a sponsor reads.

Line	Your numbers
Task (one sentence)
Model
Tokens in, per item
Tokens out, per item
Items per month
$ per item
$ per month
Per-outcome metric ($ per ticket resolved, per filing summarized, per question answered)

If the model cost is a small fraction of the value each outcome produces, the feature pays for itself. If it is not, the levers above are where you look before abandoning it.

--- title: "Appendix C: Cost and unit economics" jupyter: gaba-core --- Every applied chapter ended with a cost callout in unit-economics terms. This appendix gathers the cost discipline in one place: how to attribute cost, the levers that move it, and how to present it to someone who controls a budget. ## Translate cost per token into cost per outcome A price of "$0.40 per million output tokens" means nothing to a business sponsor. Translate it into the unit they care about: cost per ticket triaged, per document summarized, per question answered. That is the number that compares against the manual cost it replaces. ```{python} from dotenv import load_dotenv load_dotenv() from gaba.llm import cost_estimate, MODEL_DEFAULT # A triage call: a short prompt in, a one-word category out. per_call = cost_estimate(MODEL_DEFAULT, input_tokens=90, output_tokens=3) print(f"cost per ticket: ${per_call:.6f}") print(f"cost for 10,000 tickets: ${per_call * 10_000:.2f}") # Compare to the manual alternative. manual_per_ticket = 0.50 # e.g., 30 seconds of an agent's time print(f"manual cost for 10,000: ${manual_per_ticket * 10_000:,.0f}") ``` That contrast, a few dollars of model cost against thousands of dollars of human time, is the case for the project. It is also deliberately narrow: it counts only the inference, and a real proposal adds the build and maintenance cost, but the per-outcome framing is what makes the comparison legible. ## The levers that move cost In rough order of impact: | # | Lever | The move | Typical saving | Chapter | |---|---|---|---|---| | 1 | Context size | Pack less into each prompt: fewer RAG chunks, tighter instructions | Proportional: halving the context roughly halves input cost, which usually dominates the bill | Ch 10 | | 2 | Model choice and routing | Send the easy bulk to a cheap model; escalate only the hard cases | The price ratio between the models on whatever you downshift; the blended bill is the volume-weighted average of the two prices | Ch 2, 12 | | 3 | Prompt caching | Put the stable prefix (system prompt, instructions, examples) first | The provider's cache discount, applied to the cached share of input tokens | Ch 2 | | 4 | Batching | Queue work that is not time-sensitive on the batch API | Often around 50 percent off, but check your provider's current batch pricing | Ch 2 | | 5 | Self-hosting | Replace the API with a GPU you keep busy | Anything from negative to large; it pays only at high, steady volume past the break-even, usually when replacing an expensive model | Ch 4 | Check the levers in order. The first is free and immediate, while the last is a project with an up-front cost and a utilization condition attached. ## When to self-host or fine-tune Both are investments that pay off only at scale, and both have a break-even you can compute before committing (Chapter 4 for self-hosting, Chapter 15 for fine-tuning). The shape is the same: a fixed up-front cost repaid by cheaper per-call cost, so the question is always "do we have the volume," which we answer with the break-even arithmetic; a common mistake is to answer it by instinct. ## A cost worksheet For any feature, copy this table and fill in the right column, top to bottom; the last three rows are the ones a sponsor reads. | Line | Your numbers | |---|---| | Task (one sentence) | | | Model | | | Tokens in, per item | | | Tokens out, per item | | | Items per month | | | $ per item | | | $ per month | | | Per-outcome metric ($ per ticket resolved, per filing summarized, per question answered) | | If the model cost is a small fraction of the value each outcome produces, the feature pays for itself. If it is not, the levers above are where you look before abandoning it.