# Estimating AI Projects: Why the Demo Took a Week

*Published 2026-07-23 · Dmitry Shirokov · shirokoff.ca/blog/estimating-ai-projects*

A senior engineer builds a prototype over a long weekend. It answers questions about the company's documents, it's genuinely impressive, and by Thursday someone asks when it ships. The honest answer — roughly a quarter — sounds absurd next to a thing that visibly already works. The instinct is to say it's "not production-ready," which sounds like excuse-making. **The demo did about ten per cent of the work, and it's possible to name the other ninety and price it.**

## What the demo skipped

| Demo | Production | Share of effort |
|---|---|---|
| Curated documents in a folder | Real sources with permissions, formats, updates, deletions | **20–30%** |
| Answers looked good to the builder | Eval set, scoring harness, a quality bar someone signed | **15–25%** |
| The builder's credentials | Identity, authorization, per-user filtering, secrets | 10–15% |
| Ran on a laptop | Deployment, scaling, cost controls, monitoring, on-call | 10–15% |
| Happy-path questions | Failure handling, refusals, escalation, adversarial input | 10–15% |
| One person's judgment | Governance sign-off, risk assessment, audit evidence | 5–15%, more if regulated |
| — | Change management: training, comms, the workflow it lands in | Usually budgeted at zero |

**Data access and evaluation are typically half the project**, and the demo contains neither — not through sloppiness, but because a prototype is *defined* as the version that skips them. "Connect to SharePoint" is a sentence; behind it is permissions mapping, twelve undocumented document types, incremental sync with edits and deletions, and four hundred near-duplicate policies with no authoritative version. That's where quarters go.

## Five multipliers

**Data readiness (≤3×)** — one system with a clean API, or a wiki plus a shared drive plus twenty years of PDFs plus one person's inbox? Corpus hygiene is the biggest swing factor, and the only way to assess it is to look at actual documents, not the inventory.

**Consequence of being wrong (≤2.5×)** — an internal summarizer and a customer-facing policy assistant are different projects with the same architecture. Drives eval depth, review, governance evidence, and how many people hold a veto.

**Integration surface (≤2×)** — read-only retrieval is one project; an agent that *writes* is another, because every write needs authorization, idempotency, audit and an approval path. Count the write-capable tools; each is a small project.

**Number of decision-makers (≤2×)** — not technical, frequently the largest. Estimate it separately: it's **calendar** time, not effort, and conflating them is how dates slip while everyone is busy.

**Novelty to the team (≤1.8×)** — the second project is dramatically faster because the eval harness, deployment path, observability and failure intuition already exist. Say so explicitly; it's what makes the *next* estimate credible.

## How to estimate one

**Spend a week before committing.** Three outputs, because they convert the biggest uncertainties into observations: (1) **touch the real data** — a genuine sample from the real source with real permissions, which routinely halves or doubles the estimate; (2) **write thirty eval cases with the business** — not to run them, but to force the argument about what a good answer *is*; if they can't agree on thirty, that disagreement is your real risk and it's cheapest in week one; (3) **name every write operation** — empty list means a retrieval project, otherwise each item carries authorization, idempotency, audit and approval work.

Then estimate **as a range with the drivers named**: "eight to fourteen weeks; the spread is data access." A range with a reason invites a scoping conversation; a single number invites an argument about the number. And sequence so the riskiest thing is proven first — thirty real questions answered end-to-end on real data in week two, badly, and measured.

## Four traps

**Estimating the model and forgetting the workflow** — the system can be excellent and still fail because nobody changed the process or trained anyone. **Treating evaluation as a phase** — it's continuous, and budgeting it as a block at the end guarantees it gets cut. **Pricing the pilot and not the operation** — model spend, eval maintenance, the failures a human still handles, and an owner; a pilot with no operating budget is an orphan on day one. **Letting the demo set the quality bar** — the perceived remaining work is "make it do that for everyone"; the actual work is handling the questions nobody thought of.

## Three sentences for the sponsor

*"The demo proved the idea works. It didn't touch data access, the quality bar, or what happens when it's wrong — and those are about half the project."* · *"I can give you a number in a week, after we've touched the real data. Or a range now, and it's wide because of the data."* · *"What does this system do when it's wrong, and who notices?"* — the most useful scoping question in AI delivery, and a business question, so it belongs to the sponsor.

*Related: [AI-native readiness](https://shirokoff.ca/blog/ai-native-readiness) · [AI strategy](https://shirokoff.ca/blog/ai-strategy) · [Architecture Definition Documents and ADRs](https://shirokoff.ca/blog/architecture-definition-documents) · [AI governance frameworks](https://shirokoff.ca/blog/ai-governance-frameworks) · [Cost per resolved task](https://shirokoff.ca/blog/agent-unit-economics)*