Choose the lowest-cost model that meets the task requirements

Turn model selection into an operating decision

The cheapest token price is not necessarily the cheapest completed workflow. A small model can be economical when it reliably performs a bounded task. A stronger model can be worth its additional charge when it reduces failures or difficult review. The decision needs the same task, acceptance criteria, and cost accounting across candidates.

Start with volume, input length, output requirements, latency, data boundaries, and the people who will operate the system. Check whether ordinary rules or parsers already solve the predictable portion. Reserve generation for the part that benefits from it.

Build a task-specific comparison

Use a held-out set covering common work, exceptions, missing facts, and distracting instructions. For extraction, score known fields and unsupported values. For drafting, review commitments and factual grounding. For retrieval, check the cited evidence and whether the model abstains when the answer is absent.

Candidate	Useful starting hypothesis	What must be tested
Rules or deterministic parsing	Stable formats and explicit conditions	Coverage of real variations
Small local open-weight model	Bounded classification, extraction, drafting	Quality, memory and operational burden
Larger local model	Harder tasks within local data boundaries	Incremental quality versus resource cost
Frontier API	Complex reasoning or specialised modalities	Task benefit, provider charges and data policy
Adapted small model	Repeated narrow work with stable requirements	Improvement over the prompted base and maintenance cost

These are candidate roles, not guarantees. Run the comparison before fixing the architecture.

Route work with explicit rules

A practical router can use task type, input size, validation failures, source availability, and approved risk classes. Keep those conditions in application code. Do not treat an uncalibrated model confidence score as sufficient authority to perform an action.

predictable input → deterministic processing
bounded task → small model → validation → review or accepted output
failed or complex task → permitted escalation → stronger model → validation
consequential action → accountable approval

Limit retries. An invalid response should not trigger an unlimited cycle of increasingly expensive models. Record why work escalates so the team can fix recurring causes rather than merely paying for them.

Reduce avoidable work first

Retrieve relevant passages instead of resending a whole corpus. Bound output length. Cache stable results using model, prompt, source revision, and permission-aware keys. Batch non-urgent work where the operating pattern makes sense. Reuse connections and avoid repeated model loading when measuring normal operation.

A cache requires invalidation and access controls. A cheaper response that exposes another user’s information or uses a withdrawn policy has failed the workflow.

Account for the complete cost

Use a transparent worksheet:

API cost = input tokens × input rate + output tokens × output rate
local cost = hardware allocation + electricity + administration
workflow cost = inference + retrieval + integrations + review + failure handling
accepted-task cost = workflow cost / accepted tasks

Token rates may be expressed per million tokens; align units before calculating. Include retries, tool charges, and provider-specific caching rules where relevant. Date every quoted price and identify the exact model. Keep illustrative assumptions separate from measured utilisation and staff time.

For already-owned hardware, show both marginal running cost and an allocated-cost view. Low utilisation can change the conclusion. Compare low, normal, and peak volumes rather than reporting a single break-even number as a universal result.

Make adaptation earn its maintenance

The delivery-note lab compares the original student with an adapted student and a teacher. Adaptation adds data preparation, validation, training, versioning, and regression testing. A cheaper deployed model can still be the wrong choice if those costs outweigh the task-level benefit.

Keep a known-good baseline and review model choice after meaningful changes in workload or provider terms. Sustainable AI is a repeatable decision process with clear ownership, not a commitment to a single model family.

References

• Ollama model serving and resource behaviour
• xAI models and pricing
• MLX-LM
• Task-level benchmarking