OPERATING ECONOMICSTHE INFRASTRUCTURE QUESTION
When is self-hosting
actually cheaper?
A hosting invoice is one line. Utilisation, review, engineering capacity and incident ownership complete the picture.
Read the analysisTHE SPECIALIST AI MODEL RESEARCH DESKEvaluation · Economics · Ownership
FOR AI RESEARCH & ENGINEERING TEAMS
Test the case
for a smaller
model.
Evidence on model adaptation, evaluation and operating cost. Understand which findings apply to your workload before you commit.
Read the primary studies, challenge the conclusion and build a cited brief for your next experiment.
Published findings. Explicit limitations. A testable next step.Our method
BEHIND EVERY MODELCompute is a cost.
Accepted work is the outcome.
Accepted work is the outcome.
START WITH THE WORK
Read the workload brief One accepted document. Every required field.
THE RESEARCH TO START WITHEvidence synthesis · Edition 1.0Smaller can win.
Smaller can win.
The conditions matter.
When can an adapted model outperform a frontier API? Read the evidence, the limits and the experiment still needed on your workload.
Read the flagship analysisFROM THE ANALYSIS7 PRIMARY STUDIES
A benchmark win is a starting point for a test.
Conditional
Training method, retained capabilities and inference budget can change the outcome. [S3] [S5]
Still to test
Build a cited evidence brief Your current API baseline, unseen tasks, review burden and complete operating cost.
Selected studies, 2023–2025. Source links reviewed 16 October 2026.
WHAT ARE YOU TRYING TO ESTABLISH?Start with a question.
Start with a question.
Keep room to change your mind.
Our reading of the selected studies
Would a smaller adapted model meet our quality requirement?
Supported in specific published settings. Applicability to your workload remains untested.
Explore this evidenceAPPLY THE RESEARCH / YOUR BASELINEGive the baseline
Give the baseline
a fair fight.
Beat the API you would actually deploy. Record a dated, reasonably optimised alternative before paying to replace it.
COMPARATOR PROTOCOLNot established
The same document. The same acceptance policy.One protected evaluation. Two credible paths.
CURRENT SETUPYour production
Your production
configuration
Model, prompt, retrieval and the scoring policy you use today.
PRACTICAL ALTERNATIVEAn optimised
An optimised
API baseline
Reasonable prompting, caching, retrieval and routing tests.
Without a dated configuration, the baseline stays unknown. It never becomes zero.
DATA & INDEPENDENT EVALUATIONKeep the test
Keep the test
outside
the training.
Your holdout should not know your training set. Rights, labels and representative coverage matter more than record count alone.
Check data readinessThe family resemblance problem.
Synthetic exampleA new record ID can still contain a familiar customer template. Select the example to see why the split matters.
TrainingLearn from these
A
B
C
ValidationIterate against these
D
E
Protected holdoutReserve these for the decision
F
G
Source groups, template families and time boundaries need an explicit separation policy.
Metadata cannot establish semantic duplication or full-dataset quality.Inspect your manifest
ARCHITECTURE × RESPONSIBILITYChoose the intervention.
Choose the intervention.
Then choose
who owns it.
The technique and the delivery team are separate decisions. Keep them separate when comparing offers.
Find my next experimentAN EQUALLY VALID OUTCOMEDo not commission yet.
No rights, usable evaluation or accountable owner? Resolve that first.
INTERVENTION
Retain APISame protected acceptance test
TOOL / RESPONSIBILITY BRIEF
Use a managed service.
The provider operates the contracted service. Your team integrates it into the product.
- Buyer retains
- You retain evaluation, data permissions and the production decision.
- Capacity needed
- Integration and product engineering
- Working artefacts
- Configuration, evaluation and a tested fallback
EXPLORE THE METHOD / INTERACTIVE EXAMPLEThe cheapest run
The cheapest run
can still fail the task.
Quality first. Economics second.
Explore how the quality gate changes a decision. These configurations are synthetic teaching examples.
EXPERIMENT WORKSPACE
Model experiment sweep
Baseline pinnedIllustrative API comparator
API baselineSelectedAlternative×Below gate
SELECTED CONFIGURATION · SP-01Adapted specialist
$0.0823per accepted task
96%acceptance
6 of 6 configurations visible. Pinned API comparator always retained.
FAILURES & PROVENANCE
Synthetic: 40 of 1,000 documents rejected; unfamiliar template families.
Dataset: TF-FIXTURE-01 · 1,000 synthetic cases · no measured uncertainty. All configurations use the same illustrative scoring policy.Meets example quality gate
Deployment: Private
Rights: Confirmed
p95 latency: 1200 ms (assumed)
Not tested: your production task, your protected holdout and your baseline. Define the pilot
THE COMMERCIAL TESTCount the work
Count the work
that gets accepted.
All recurring operating costsAccepted completed tasksCOST PER ACCEPTED TASK
Synthetic scenario
Change the assumptions.
Other components stay fixed to the example. The full calculator includes review, hosting, engineering and downside scenarios.
Open the full calculatorExample quality gate metAssumed performance
Practical API$0.0957per accepted task
Specialist configuration$0.0823per accepted task
MONTHLY CASH SAVING$1,100
SEPARATE INVESTMENT$60,000
SIMPLE PAYBACK54.5 months
Before growth, discounting and tax. Accepted tasks: API 94,000, specialist 96,000. No model tested.
THE QUESTION BEHIND THE SAVING
Will it still make sense with less volume, more human review and a cheaper API?
Test the downsideDECISION GUIDES & WORKING FRAMEWORKSPut the findings
All research & field notes Put the findings
to work.
OPERATING ECONOMICSWhen is self-hosting
FOUR DECISIONS TO GET RIGHTFIELD NOTES
01Economics
02Fine-tuning vs API cost: compare accepted tasks, not tokens.
A complete cost comparison starts with the work your customer actually accepts.
Architecture
03RAG vs fine-tuning for proprietary data.
Distinguish missing knowledge from stable behavioural errors.
Data & evaluation
04Preventing customer-template leakage in extraction evaluations.
A new record ID can still contain a familiar template.
Procurement
A buyer-controlled specialist-model pilot brief.
Approve the comparator, gates and artefacts before the experiment.
THE COMPLETE BUYER’S GUIDEOne brief.
One brief.
A more informed
commitment.
Take the method into your next technical or commercial review. A practical guide to evaluation, costs, suppliers and the right to walk away.
INSIDE THE GUIDE
Read a chapter first Instant download with sources. No email required.Baseline protocol · Data readiness · Pilot gates
Supplier obligations · Ownership & exit
WHO COULD DO THE WORK?Start with the scope.
Compare them on the same terms
Start with the scope.
Then the supplier.
Explore the landscape through responsibilities and evidence. Inclusion does not imply endorsement; current provider scope requires verification.
Explore the full directory Have proposals already?Compare them on the same terms
REFERENCE ORGANISATIONCATEGORY
Together AI
Adaptation platformsFireworks AI
Adaptation platformsPredibase
Adaptation platformsGretel
Synthetic-data providersGiskard
Evaluation specialistsPatronus AI
Evaluation specialistsLandscape references from the supplied brief. No paid ranking or verified partnership.
INDEPENDENT BY DESIGNThe best recommendation
The best recommendation
may be to keep the API.
Buyer-funded diligence is the intended model. Clear evidence should support the next decision—even when that decision is to stop.