Appendix E — Appendix E: Vendor and model selection

In Chapter 15 we made the case that model choice in 2026 is a portfolio, because no single model serves every task well. In this appendix we outline the fuller rubric of how to choose which models to use, host, or fine-tune.

E.1 Closed frontier versus open weights

The first split is whether you call a model over an API or run the weights yourself.

Closed frontier models (the strongest offerings from the major labs) lead on raw capability and are the easiest to start with: an API key is all it takes to begin. The trade-offs are that your data goes to the provider, you cannot host them yourself, you cannot fine-tune most of them deeply, and the provider can change or retire the model under you.

Open-weight models (Llama from Meta, Mistral, Qwen from Alibaba, DeepSeek, etc.) are models whose trained weights are released, though the training data and code may remain private, so open-weight is not the same as fully open-source. Because the weights are available, you can download, host, and fine-tune them. As of 2026, the capability gap to the frontier has narrowed substantially on many business tasks, though it varies by task and model, so the reasons to choose open-weight models are increasingly about control: your data never leaves your systems, you can fine-tune the weights, and no one can change the model under you.

E.2 The factors that actually decide it

Benchmark scores rarely make the decision. These do:

Data residency. If your data cannot leave your walls, the choice is made for you: an open-weight model you host. This single factor overrides all others when it applies (Appendix D).
License. “Open-weight” licenses differ sharply. Some permit any commercial use, while others restrict it (by company size, use case, or whether you can train on the outputs). Read the license against your actual intended use before committing, because the wrong license is a legal problem that no technical fix can repair.
Where it can run. Some models are gated, region-locked, or large enough to need specific hardware. Confirm you can actually run the model where you need it.
Cost at your volume. Cheap per-call API models almost always beat self-hosting until volume is very large (Chapter 4). Premium models change that math sooner.
Capability on your task. Measured on your data (Chapter 9 and Appendix B), because public leaderboard scores come from tasks other than yours.

E.3 The portfolio pattern

The mature answer is usually a routed portfolio, because a single model seldom fits every kind of call:

a cheap, fast model for the high-volume easy work (most of your calls);
a stronger model, escalated to only for the hard cases that need it (Chapter 2’s routing, Chapter 12’s agents);
an open-weight model you host for anything touching regulated or residency-restricted data;
a small fine-tuned model where a narrow high-volume task makes the economics work (Chapter 15).

Each model does the work it fits. The mistake is forcing one model to do everything: the cheap one fails the hard cases, or you pay premium prices for trivial ones.

E.4 Switching costs

Selection is not a one-time decision, because vendors change under you, so weigh how hard each choice is to undo. Although the language model draws the most attention, the stickiest commitment in the stack is usually the embedding model: every vector in your database came from it, so changing it means re-embedding the entire corpus and reindexing, a real bill and a real migration (Chapter 6). Language models are easier to swap but not free of risk: providers deprecate models on their own schedules, so check the deprecation policy and notice period before building on one. The standard mitigation for the model layer is a gateway or router (OpenRouter, which this book itself runs on, or LiteLLM) that presents one API across providers, so swapping a model requires only a configuration change where it would otherwise require a rewrite. Although a gateway does not solve embedding lock-in, it keeps the rest of the portfolio movable.

E.5 A selection checklist

For any model you are considering, before adopting it: Can your data legally go to it? Does its license permit your use? Can you run it where you need to? What does it cost per outcome at your volume, and how does it score on your own evaluation set? If you can answer those five, you can defend the choice, which is more than a benchmark ranking ever lets you do.

--- title: "Appendix E: Vendor and model selection" --- In Chapter 15 we made the case that model choice in 2026 is a portfolio, because no single model serves every task well. In this appendix we outline the fuller rubric of how to choose which models to use, host, or fine-tune. ## Closed frontier versus open weights The first split is whether you call a model over an API or run the weights yourself. **Closed frontier models** (the strongest offerings from the major labs) lead on raw capability and are the easiest to start with: an API key is all it takes to begin. The trade-offs are that your data goes to the provider, you cannot host them yourself, you cannot fine-tune most of them deeply, and the provider can change or retire the model under you. **Open-weight models** (Llama from Meta, Mistral, Qwen from Alibaba, DeepSeek, etc.) are models whose trained weights are released, though the training data and code may remain private, so open-weight is not the same as fully open-source. Because the weights are available, you can download, host, and fine-tune them. As of 2026, the capability gap to the frontier has narrowed substantially on many business tasks, though it varies by task and model, so the reasons to choose open-weight models are increasingly about control: your data never leaves your systems, you can fine-tune the weights, and no one can change the model under you. ## The factors that actually decide it Benchmark scores rarely make the decision. These do: - **Data residency.** If your data cannot leave your walls, the choice is made for you: an open-weight model you host. This single factor overrides all others when it applies (Appendix D). - **License.** "Open-weight" licenses differ sharply. Some permit any commercial use, while others restrict it (by company size, use case, or whether you can train on the outputs). Read the license against your actual intended use before committing, because the wrong license is a legal problem that no technical fix can repair. - **Where it can run.** Some models are gated, region-locked, or large enough to need specific hardware. Confirm you can actually run the model where you need it. - **Cost at your volume.** Cheap per-call API models almost always beat self-hosting until volume is very large (Chapter 4). Premium models change that math sooner. - **Capability on your task.** Measured on your data (Chapter 9 and Appendix B), because public leaderboard scores come from tasks other than yours. ## The portfolio pattern The mature answer is usually a routed portfolio, because a single model seldom fits every kind of call: - a cheap, fast model for the high-volume easy work (most of your calls); - a stronger model, escalated to only for the hard cases that need it (Chapter 2's routing, Chapter 12's agents); - an open-weight model you host for anything touching regulated or residency-restricted data; - a small fine-tuned model where a narrow high-volume task makes the economics work (Chapter 15). Each model does the work it fits. The mistake is forcing one model to do everything: the cheap one fails the hard cases, or you pay premium prices for trivial ones. ## Switching costs Selection is not a one-time decision, because vendors change under you, so weigh how hard each choice is to undo. Although the language model draws the most attention, the stickiest commitment in the stack is usually the embedding model: every vector in your database came from it, so changing it means re-embedding the entire corpus and reindexing, a real bill and a real migration (Chapter 6). Language models are easier to swap but not free of risk: providers deprecate models on their own schedules, so check the deprecation policy and notice period before building on one. The standard mitigation for the model layer is a gateway or router (OpenRouter, which this book itself runs on, or LiteLLM) that presents one API across providers, so swapping a model requires only a configuration change where it would otherwise require a rewrite. Although a gateway does not solve embedding lock-in, it keeps the rest of the portfolio movable. ## A selection checklist For any model you are considering, before adopting it: Can your data legally go to it? Does its license permit your use? Can you run it where you need to? What does it cost per outcome at your volume, and how does it score on your own evaluation set? If you can answer those five, you can defend the choice, which is more than a benchmark ranking ever lets you do.