The client had two people and a phone that rang about forty times a week. Roughly a third of those calls went to voicemail — during a job, after five, on a Saturday — and a meaningful share of the voicemails never turned into anything, because someone who wants an appointment on Saturday morning calls the next place on the list before you call back on Monday. They didn't want a call centre. They wanted the phone answered.
That is a very specific and very common problem, and it is now solvable in a couple of weeks by one person. It is also full of small traps that turn a great demo into a system nobody trusts — most of them not in the AI at all, but in the phone number, the calendar, and what happens on the fourth turn of a conversation the demo never got to.
This is the build log: the platform decision, getting a real Canadian number and what Canada specifically requires, prototyping the business logic in n8n so it can change daily, wiring Google Calendar without double-booking anyone, and a testing method that catches what a laptop demo cannot. Everything as of late 2025, and the vendor landscape here moves fast enough that you should treat version-specific details as directional.
What a receptionist actually has to do
Write this down before you touch a platform, because it's the spec and everyone skips it. The job is not "have a conversation." It's a small set of outcomes with a hard requirement on each:
| Outcome | Hard requirement |
|---|---|
| Book an appointment | Real availability from the real calendar. Never a guess, never a "typically we have mornings free." |
| Reschedule or cancel | Find the existing booking from a caller who does not know its ID |
| Answer a factual question (hours, address, parking, price range) | From a maintained source, verbatim where it matters. No improvisation. |
| Take a message | Name, callback number, reason — captured accurately enough to act on |
| Escalate | A defined number, defined conditions, and context carried across |
| Say it doesn't know | Cleanly, then do one of the above — this is a feature, not a failure |
Six outcomes. Everything else the caller might say routes into "take a message" or "escalate." The single biggest quality lever in this whole build is narrowing the job to those six and making the agent decline the rest gracefully, because an agent that tries to be helpful about something outside its remit is an agent that invents a policy.
Choosing a layer, not a vendor
There are three layers you can build at, and the mistake is arguing about vendors before deciding which layer you're on.
| Layer | Examples | You write | Right when |
|---|---|---|---|
| Managed agent platform | ElevenLabs Agents, Vapi, Retell | A prompt, tool definitions, and the tools themselves | Almost always, for a receptionist. Telephony, turn-taking and latency are handled. |
| Voice framework | LiveKit Agents, Pipecat | The pipeline: VAD, ASR, LLM, TTS, interruption handling | You need control the platform won't give — custom models, unusual media routing |
| Direct realtime API | OpenAI Realtime and peers | Everything, including the phone bridge | You are building the platform, not the receptionist |
For this job I used a managed platform, and I'd do it again without much deliberation. The hard parts of a voice agent — knowing when the caller has finished speaking, handling interruptions, keeping the round trip under the threshold where a caller starts talking over you — are solved products at that layer, and they are genuinely difficult to reproduce. Reach for a framework when you have a requirement the platform can't express, not because building it yourself feels more legitimate.
I used ElevenLabs Agents because the voice quality is a real differentiator on a phone line, it has a native Twilio integration and SIP trunking so telephony is configuration rather than code, and its tool-calling maps cleanly onto the six outcomes. Vapi and Retell solve the same shape; the decision between them is mostly about voice quality, telephony ergonomics and per-minute price, and it is worth an afternoon of side-by-side calls rather than a spreadsheet.
Getting a real Canadian number
The part everyone underestimates, so here it is concretely.
Buy the number from a telephony provider, not from the agent platform, if you ever want to move. A number bought inside a platform is convenient and one more thing you'd have to port later. I used Twilio, bought a local number in the client's own area code — which matters more than it sounds, because a familiar area code gets answered and a toll-free one reads as a call centre.
What Canada specifically requires:
- A verified local address. Canadian numbers need an address on file, and providers reject P.O. boxes and virtual addresses. Have the client's actual business address and a document that proves it before you start, or you'll lose a day.
- Caller ID authentication is the carrier's job, not yours. Canadian carriers have been required to implement STIR/SHAKEN since November 2021, which is why a well-attested number gets through and a badly-configured one gets flagged. Practically: use a reputable provider, keep the number's registration accurate, and don't spoof caller ID on outbound.
- Keeping an existing number. Two options — port it (clean, slow, and a scheduled cutover) or point the existing line at the agent over SIP trunking, which most platforms now support. SIP is the low-risk path for a business that cannot afford a porting window, and it's how I'd start: forward after hours first, port later if ever.
- Emergency calling. A VoIP number is not a 911 line for the business. Say this out loud to the client and put it in writing; it is the kind of assumption that goes unexamined until it matters.
- Outbound is a different legal regime. Inbound answering is straightforward. The moment the agent places calls you are in CRTC unsolicited-telecommunications territory, with the National Do Not Call List and consent rules in play. Appointment reminders to existing customers are generally fine; anything resembling solicitation needs advice, not a blog article.
Two disclosure rules I now apply by default in Canada. Say it's an AI in the first sentence — it costs nothing, callers are fine with it when it works, and discovering it later is what makes people angry. And if you record or transcribe, say so: the recording itself is one-party-consent territory, but privacy law expects people to be told when their personal information is being collected, and a transcript of a phone call is personal information. One sentence in the greeting covers both.
The architecture
graph TB CALLER["Caller"] -->|"PSTN"| TW["Twilio number
local area code, verified address"] TW -->|"native integration / SIP"| AGENT["Voice agent platform
ASR + LLM + TTS + turn-taking"] KB[("Knowledge base
hours, address, services, prices")] --> AGENT AGENT -->|"tool: check_availability"| N8N{"n8n webhook
business logic"} AGENT -->|"tool: book_appointment"| N8N AGENT -->|"tool: take_message"| N8N N8N -->|"freebusy + insert"| GCAL[("Google Calendar")] N8N -->|"confirmation"| SMS["SMS + email"] N8N -->|"row per call"| SHEET[("Log: bookings, messages")] AGENT -->|"transfer on escalation"| HUMAN["Owner's mobile
with context"] AGENT -->|"post-call webhook"| POST["Transcript · outcome
· extracted fields"] POST --> REVIEW["Daily review queue
every failure becomes a test case"]
Four things worth noticing. The knowledge base is separate from the prompt, so the client can change their hours without anyone editing an agent. All business logic sits behind three tools rather than in the prompt, which is what keeps the agent from inventing availability. The post-call webhook is the whole feedback loop — transcript, outcome and extracted fields on every call, feeding a review queue. And escalation is a real path with a real number, not a phrase the agent says.
Prototyping the logic in n8n
Here's the argument for n8n on a build like this, and it isn't "low-code is easier." In the first two weeks the business logic changes almost daily — the buffer between appointments, which service types can be booked at all, what happens when someone asks for a slot on a statutory holiday, whether a new customer needs a longer first appointment. Every one of those is a five-minute change in a workflow and a deploy cycle in application code, and crucially the client can watch it change in front of them.
The contract between the agent and n8n is a webhook per tool. Keep it boring and strict:
// Tool definition given to the agent. Two rules earn their keep here:
// the description tells the model WHEN to call it, and the schema is narrow
// enough that there is very little for it to get creative about.
{
"name": "check_availability",
"description": "Check real appointment availability. Call this before offering ANY time to the caller. Never state availability that did not come from this tool.",
"url": "https://n8n.example.ca/webhook/check-availability",
"method": "POST",
"parameters": {
"type": "object",
"properties": {
"service_type": { "type": "string", "enum": ["consult", "standard", "followup"] },
"preferred_date": { "type": "string", "description": "ISO date, e.g. 2025-12-03" },
"part_of_day": { "type": "string", "enum": ["morning", "afternoon", "any"] }
},
"required": ["service_type", "preferred_date"]
}
}
// The response shape matters as much as the request. Return SPEAKABLE strings,
// not raw timestamps — otherwise the model reformats them and occasionally
// reformats them wrong.
{
"available": true,
"slots": [
{ "id": "2025-12-03T09:30-05:00", "say": "Wednesday the third at nine thirty in the morning" },
{ "id": "2025-12-03T14:00-05:00", "say": "Wednesday the third at two in the afternoon" }
],
"next_available_say": "Wednesday the third at nine thirty in the morning"
}Two details in there are the difference between a system that works and one that mostly works. Returning a say string per slot stops the model translating an ISO timestamp into speech, which it does well ninety-something per cent of the time — and the failures are the memorable kind, like offering "fourteen o'clock." And the opaque id means booking passes back exactly what the availability check produced, so there's no window for a hallucinated time to reach your calendar.
The booking tool, and the confirmation rule
One rule, non-negotiable: the agent must repeat the details back and get a yes before it books. Name, service, day, time. It costs one conversational turn and it eliminates the entire class of failure where a misheard date becomes a real calendar entry that a real person then has to unpick.
On the n8n side, book_appointment does five things in order: re-check the slot is still free (someone may have booked it in the ninety seconds since the availability call), create the calendar event with the caller's details in the description, write a row to the log, fire the SMS and email confirmation, and return a short confirmation string for the agent to speak. If step one fails, return the failure honestly with two alternatives rather than an error the model has to improvise around.
Google Calendar, and the four traps
Timezones. Store and compute in the business's timezone with a proper zone identifier — America/Toronto, not a fixed offset — and never let the model do arithmetic on it. Canada's DST transitions will produce a wrong appointment exactly twice a year, always on a weekend, and always in a way that looks like the agent hallucinated.
Free/busy is not the whole answer. A free/busy query tells you where existing events are; it doesn't know your business hours, your lunch break, the buffer you want between jobs, or that Mondays are reserved for something else. Generate candidate slots from a rules layer in n8n, then subtract busy time. Doing it the other way round is how the agent offers an appointment at 7:40pm on a Sunday.
The double-booking race. Two callers, or a caller and the owner's own phone, can take the same slot in the same minute. Re-check at booking time and — if the stakes justify it — hold a tentative event during the conversation and confirm or delete it. Cheap insurance against the failure that most damages trust.
Auth and ownership. Use a service account with access granted to the specific calendar, not one employee's personal OAuth token, or the day that person changes their password the receptionist stops working. Grant the narrowest scope that permits reading availability and creating events.
Testing a voice agent
This is where most builds are weakest, because the demo is so convincing. Three layers, and you need all three.
1. Simulated conversations, in CI
Modern platforms can run a scripted or LLM-driven simulated caller against your agent end to end and evaluate whether the conversation reached a defined outcome — ElevenLabs exposes exactly this as a simulation API with configurable evaluation criteria. Write one scenario per outcome plus one per known failure, and run them on every prompt or tool change. It is text-level and fast, and it catches logic regressions: the agent that stopped offering to take a message, the agent that started booking without confirming.
2. Real calls from a real phone
Non-negotiable, and the one people skip because it isn't automatable. The audio path on a phone call is nothing like your laptop microphone — narrowband codec, packet loss, background noise, a caller on speakerphone in a car. Behaviour that is flawless in the browser widget degrades measurably on the PSTN, and there is no substitute for making the calls. My standing matrix:
| Test call | What it exposes |
|---|---|
| Speakerphone in a moving car | The realistic worst case for ASR; where endpointing falls apart |
| Spell an unusual surname | Name capture — the most common accuracy failure by far |
| Give a phone number quickly, once | Digit accuracy; whether the readback catches an error |
| "Next Tuesday" / "the week after next" | Relative-date resolution, and whether it confirms rather than assumes |
| Interrupt mid-sentence | Barge-in handling; does it stop talking and actually listen |
| Two accents, one non-native speaker | The gap between your test set and your callers |
| Silence for ten seconds | Does it prompt gracefully or talk into the void |
| Ask something out of scope, insistently | Whether "I don't know, let me take a message" survives pressure |
| Ask for a human immediately | The escalation path, end to end, including context transfer |
| Hang up mid-booking | No orphaned calendar events, no half-written rows |
3. Production review, every day at first
Configure the post-call webhook to deliver the transcript, the outcome, and structured fields extracted from the conversation, then read every call for the first week and a sample thereafter. This is where you learn what people actually ask — which is never what you guessed — and every failure becomes a simulation case, the same discipline as building a golden eval set from production traffic. Two metrics worth a dashboard from day one: containment (calls resolved without escalation) and booking accuracy (appointments that the human didn't have to fix), because those are the two numbers the client actually cares about.
The lessons that cost me. Latency is a product feature and the tool call is where it goes. A conversational turn feels natural under roughly a second and a half; a cold n8n workflow plus a Calendar round trip can eat all of it and callers start talking over the silence. Two fixes: have the agent speak a short acknowledgement — "let me check that for you" — while the tool runs, and pre-compute the next few days of availability so the common case is a lookup rather than an API call. Don't let the model format times. Return speakable strings from the tool; the day it says "fourteen o'clock" to a customer is the day the client loses confidence in the whole thing. Test on the actual phone network early, not after the browser demo is perfect — I lost two days polishing a prompt against problems that turned out to be codec artefacts. Scope creep is the real risk. Every client asks whether it can also take payments, answer detailed pricing questions, handle complaints. Each addition widens the surface where it can be confidently wrong, and the value is concentrated in booking and messages. And ship it after hours first. Point the number at the agent from 5pm and on weekends, leave business hours with the humans, and let the client hear a week of real recordings before it answers a single call during the day. That sequencing has done more for adoption than any technical decision on this page.
What it costs, roughly
Three stacked per-minute costs — telephony, the voice platform (which bundles ASR, LLM and TTS), plus whatever your automation layer charges — and for a small business the total lands in the range of a modest monthly subscription rather than a salary, dominated by the platform's per-minute rate. The useful comparison isn't against a receptionist's wage; it's against the bookings currently going to voicemail. For the client I opened with, recovering a handful of after-hours bookings a month covered it several times over, and that arithmetic is the entire business case — as with any agent, the number that matters is cost against the outcome, not cost per minute.
What to carry away
Define the job as six outcomes and make the agent decline everything else gracefully — narrowing scope is the biggest quality lever available. Build on a managed voice platform unless you have a requirement it can't express; turn-taking and latency are solved products and hard to rebuild.
Get the number right: buy it from a telephony provider, use a local area code, have a real verified address ready (no P.O. boxes), start by forwarding over SIP rather than porting, and be explicit with the client that a VoIP line is not an emergency line. Disclose the AI in the first sentence and disclose recording — one sentence covers both, and it prevents the reaction that no amount of accuracy recovers from.
Put the business logic in n8n behind two or three narrow tools while it's still changing daily, return speakable strings and opaque slot ids so the model never formats a time or invents one, and require a spoken confirmation before anything is written to a calendar. Generate slots from business rules and then subtract free/busy, re-check at booking time, and use a service account.
Then test in three layers — simulated conversations in CI, a matrix of real calls from a real phone including the car-on-speakerphone case, and daily review of production transcripts where every failure becomes a new test. Launch after hours, watch containment and booking accuracy, and expand into business hours only once the client has heard a week of it working.