A senior engineer builds a prototype over a long weekend. It answers questions about the company's documents, it's genuinely impressive, and it gets demoed to the executive team on Tuesday. By Thursday somebody has asked the only question that matters — when can we ship it? — and the honest answer, roughly a quarter, sounds absurd next to a thing that visibly already works.
This is the most common credibility problem in AI delivery, and I've been on both sides of it. The instinct is to explain that the demo is "not production-ready," which sounds like engineering excuse-making. The better move is to be specific: the demo did about ten per cent of the work, and it's possible to name the other ninety and price it. This article is that breakdown, the multipliers that move an estimate by 3×, and a way to give ranges you can actually defend.
What the demo skipped
A prototype is a system with every hard requirement removed. The removals are invisible in the demo, which is why the estimate feels arbitrary. Naming them individually turns a vague "it's harder than it looks" into a list a sponsor can reason about.
| What the demo did | What production requires | Typical share of total effort |
|---|---|---|
| Curated documents in a folder | Real sources, with permissions, formats, updates and deletions | 20–30% |
| Answers looked good to the builder | An evaluation set, a scoring harness, a quality bar someone signed | 15–25% |
| The builder's credentials | Identity, authorization, per-user filtering, secrets handling | 10–15% |
| Ran on a laptop | Deployment, scaling, cost controls, monitoring, on-call | 10–15% |
| Happy-path questions | Failure handling, refusals, escalation, adversarial input | 10–15% |
| One person's judgment | Governance sign-off, risk assessment, audit evidence | 5–15%, more if regulated |
| — | Change management: training, comms, the workflow it lands in | Frequently underestimated at zero |
Read the first two rows together and you have most of the answer. Data access and evaluation are typically half the project, and the demo contains neither. Not because the prototype was sloppy — because a prototype is defined as the version where you skip those.
The data row is the one that surprises sponsors most and engineers least. "Connect to SharePoint" is a sentence; the work behind it is permissions mapping, format handling for the twelve document types nobody mentioned, an incremental sync that handles edits and deletions, and a conversation about the four hundred documents that turn out to be duplicates of each other with different dates. That's where quarters go.
The five multipliers
Two AI projects with identical descriptions can differ by 3× in effort. Five factors explain almost all of it, and each is knowable before you estimate — which is what makes an estimate defensible rather than a guess.
1. Data readiness (up to 3×)
Is the content in one system with a usable API and clean permissions, or spread across a wiki, a shared drive, twenty years of PDFs and one person's inbox? Are documents current, or does the corpus contain three versions of every policy with no way to tell which is authoritative? Corpus hygiene is the single biggest swing factor, and the only reliable way to assess it is to look at the actual documents rather than the inventory.
2. Consequence of being wrong (up to 2.5×)
An internal summarizer that saves ten minutes and an assistant that quotes policy to customers are different projects with the same architecture. Consequence drives evaluation depth, review requirements, governance evidence and the number of stakeholders who get a veto. This is the axis I'd tier on for governance too, and it's the one that most changes who has to approve.
3. Integration surface (up to 2×)
Read-only retrieval over documents is one project. An agent that writes — creates a ticket, issues a refund, updates a record — is another, because every write needs authorization, idempotency, an audit trail, and a human approval path for the irreversible ones. Count the write-capable tools; each one is a small project.
4. Number of decision-makers (up to 2×)
Not a technical factor and frequently the largest one. A project with one accountable owner moves; a project needing legal, security, three business units and a works council to agree on what "good" means will spend more calendar time in alignment than in engineering. Estimate this honestly and separately — it's calendar time, not effort, and conflating the two is how dates get missed while everyone is busy.
5. Novelty to the team (up to 1.8×)
A team that has shipped one of these before moves dramatically faster, because they already own the eval harness, the deployment path, the observability wiring and — most valuable — the intuition for which failures matter. The first one is always the expensive one, and it is worth telling a sponsor that explicitly, because it makes the second project's estimate credible.
graph LR DEMO["Demo
1 week"] --> BASE{"Base build
3-5 weeks"} BASE --> DATA["+ data access
×1 to ×3"] DATA --> RISK["+ consequence
×1 to ×2.5"] RISK --> INT["+ write integrations
×1 to ×2"] INT --> DEC["+ decision-makers
×1 to ×2 (calendar)"] DEC --> NOV["+ team novelty
×1 to ×1.8"] NOV --> EST["Estimate as a RANGE
with the drivers named"] EST -.->|"re-estimate after
the data spike"| DATA
Estimate the base build, then apply the multipliers you can actually observe — and present the multipliers, not just the total, because a sponsor who can see that "×2.5 for customer-facing consequence" is in the number can choose to change the scope instead of arguing with the date. The dotted line is the important discipline: the data multiplier is a guess until someone has spent a week in the real corpus, so the first estimate is provisional by design.
How I actually estimate one
Spend a week before you commit to a number. Not a design phase — a spike with three specific outputs, because these three convert the largest uncertainties into observations:
- Touch the real data. Pull a genuine sample from the actual source with the actual permissions. You will discover the eight document formats, the duplicates, the authorization model nobody documented, and whether "we have all our policies in SharePoint" is true. This alone routinely halves or doubles the estimate.
- Write thirty evaluation cases with the business. Not to run them — to force the conversation about what a good answer is. If the business can't agree on the right answer to thirty questions, that disagreement is your project's real risk, and it's much cheaper to surface it in week one than in UAT.
- Name the write operations. List every action the system takes that changes something. If the list is empty, you have a retrieval project. If it isn't, each item carries authorization, idempotency, audit and approval work that belongs in the estimate explicitly.
Then estimate as a range with the drivers named: "eight to fourteen weeks; the spread is data access — if the permissions model is what we were told, eight; if we have to reconcile three sources of policy documents, fourteen." A range with a reason invites a scoping conversation. A single number invites an argument about the number.
And structure the plan so the riskiest thing is proven first. The traditional order — build the pipeline, then evaluate — hides the biggest risk until late. Invert it: get thirty real questions answered end-to-end on real data in week two, badly, and measure. You learn whether the corpus can support the product before you've built anything expensive on top of it, which is the same argument I make for starting with the smallest supervised version of a use case.
Four traps that turn a good estimate into a missed date. Estimating the model and forgetting the workflow. The system can be excellent and still fail because nobody changed the process it lands in, nobody was trained, and the people whose job it touches weren't consulted. That work is real and it is nobody's default responsibility. Treating evaluation as a phase. It's a continuous cost — building the set, maintaining it, re-labelling when the definition of good shifts — and budgeting it as a two-week block at the end guarantees it gets cut. Pricing the pilot and not the operation. The build is one number; running it is another, ongoing one — model spend, eval maintenance, the failures a human still handles, and the person who owns it. A pilot with no operating budget becomes an orphan the day it ships. Letting the demo set the quality bar. The prototype answered the five questions its builder thought of, and the perceived remaining work is "make it do that, but for everyone." The actual remaining work is making it handle the questions nobody thought of — which is a different and much larger job, and the one the estimate is mostly about.
Talking about it with a sponsor
Three sentences that have worked better for me than any amount of technical explanation:
"The demo proved the idea works. It didn't touch the data access, the quality bar, or what happens when it's wrong — and those are about half the project." This reframes the gap as scope rather than as engineering caution, which is both more accurate and easier to act on.
"I can give you a number in a week, after we've touched the real data. Or a range now, and the range is wide because of the data." Sponsors respect a costed uncertainty far more than a confident number that later moves. And the week is cheap.
"What does this system do when it's wrong, and who notices?" The single most useful scoping question in AI delivery. The answer determines your evaluation depth, your approval requirements and half your architecture — and it's a business question, so it belongs to the sponsor rather than to you.
What to carry away
The demo-to-production ratio isn't padding: a prototype is the version with every hard requirement removed, and data access plus evaluation are roughly half of what's left. Name the removals explicitly — real sources with permissions and deletions, an evaluation set with an agreed quality bar, identity and authorization, deployment and cost control, failure handling, governance evidence, and the change management nobody budgets.
Apply the five multipliers you can observe before estimating: data readiness (the biggest swing), consequence of being wrong, write-capable integrations, number of decision-makers (calendar time, not effort), and novelty to the team (the first one is always the expensive one).
Spend a week before committing: touch the real data, write thirty evaluation cases with the business, and list every write operation. Then give a range with its drivers named, and sequence the plan so the corpus is proven end-to-end in week two rather than validated at the end. The estimate that survives contact with delivery is the one where the sponsor can see which assumption they'd have to change to get a different date.