The Total Cost of Owning a Custom AI System

The build cost is the down payment

Teams approve a custom AI system on the build number and get surprised by everything after. Total cost of ownership for AI is the build, plus the monthly run cost, plus a set of lifecycle costs that recur for as long as the system exists — and the third bucket is the one that's usually missing from the business case.

Bucket 1: the build

The model is rarely the long pole. The weeks go into the data pipeline, the integrations and their schemas, the evaluation harness, the escalation console, and the audit logging. On our support-agent engagement the build was six weeks; the model choice was a small part of it.

Bucket 2: the run cost

• Model inference per unit of work — and a multi-step task is several calls, multiplied by volume, with a peak multiplier.
• Infrastructure — a feature store's reads, a vector index, a message broker, async serving, logging.
• Escalation staffing — the human share never goes to zero.

Bucket 3: the lifecycle costs

This is the bucket that gets forgotten:

• Retraining. Fraud patterns move, catalogues change, packaging changes. Models decay and need re-training on a cadence.
• Monitoring and on-call. Someone owns the drift alerts, the cost dashboards, the tool-error rate — and someone is paged when the model is wrong at scale.
• Evaluation-set upkeep. Every production surprise becomes a new test case; the eval set is a living artefact.
• Model migration. The model you built on gets deprecated or a better one ships. Re-testing, re-tuning prompts, sometimes re-embedding a whole catalogue.
• Prompt and policy maintenance. As the business changes, so do the instructions and the rules.

A rough rule: over three years, buckets 2 and 3 together usually exceed bucket 1.

Build vs buy on a TCO basis

A managed product moves bucket 1 to near-zero and shrinks bucket 3 (the vendor retrains and migrates), against a per-usage fee that scales with the business. Building wins on TCO when the per-usage fee at your volume, over three years, exceeds build plus run plus lifecycle — and you have the team to own bucket 3. A custom system nobody is staffed to maintain is a depreciating asset, and its real TCO includes the day it quietly stops working.

A three-year sketch

Lay it out on a timeline. Bucket 1 is a one-off at the start. Bucket 2 is the monthly run cost times 36. Bucket 3 is a retraining cadence (say quarterly), a fraction of a role for monitoring and on-call across the whole period, and a budget line for one model migration somewhere in the three years. The projects in our portfolio each carry a measured per-unit run cost — per transaction, per conversation, per search — which is the bucket-2 input; the mistake is stopping there and forgetting bucket 3 exists.

When the vendor wins on TCO

If the per-usage fee at your volume, summed over three years, comes in under build plus run plus lifecycle — and it often does at low-to-moderate volume — buying is the lower total cost, and it's the responsible choice unless you can genuinely staff the ownership. Building wins when volume, control needs, or a per-usage fee that scales with success tip the three-year sum the other way.

Where this stops being right

• A short-lived feature — if it'll be retired in a year, the lifecycle costs are small and the TCO is close to build plus a few months of run.
• A vendor whose error profile genuinely fits — buying is often the lower TCO, and it's the responsible choice without a maintenance team.
• Very low volume — the run and lifecycle costs are dominated by fixed minimums; the estimate is mostly "smallest viable footprint."

FAQ

What's the biggest hidden cost? Bucket 3 — retraining, monitoring, on-call, eval upkeep, and eventual model migration. It recurs long after the build ships.

How does build vs buy look on TCO? Buying zeroes the build and shrinks the lifecycle costs at a per-usage price. Building wins when that price at your volume over three years exceeds building and owning it — and only if you can staff the ownership.

Can we estimate lifecycle cost up front? Within a range: a retraining cadence, a fraction of a role for monitoring and on-call, and a budget line for one model migration inside three years.

ISTRALLEN sizes AI systems on total cost of ownership — build, run, and the lifecycle bucket most business cases miss — see what we do.