How we test
Methodology
Every service in a round gets the same business profile, the same call scripts, and the same scoring sheet. This page is the rulebook. We will not change the script mid-round to favor a vendor.
Published Nov. 18, 2026. First test round in progress — no results yet.
What we are testing
We are testing the product a small business would actually use: an AI or virtual receptionist that answers inbound calls, takes a message or books an appointment, and follows written rules. We are not testing outbound sales dialers as the primary product, even if a vendor also sells them.
Ruby is included as a live-human benchmark because callers still compare AI tools with traditional virtual receptionists. Ruby’s own FAQ says it is not an automated virtual receptionist.[1]
The test business
We use a clearly fictional business so we are not forwarding a real company’s phones or inventing a live shop.
Name: Cedar & Main Demo Co. (fictional)
What it does: residential plumbing and related home-service calls, plus a general appointment desk we can reuse for industry variants.
Hours: Monday–Friday, 8:00 a.m. to 5:00 p.m. Pacific Time. Closed Saturday and Sunday.
Service area: within 20 miles of Olympia, Washington.
Knowledge-base prices we will load (fictional, for scoring only): service call $89; drain clearing $189; after-hours emergency dispatch fee $149 on top of the service call. If an agent quotes a price we did not load, that is a miss.
Rules we will load:
- Emergency (active leak, no heat, sewage backup, or the caller says “emergency”): take the address and a callback number, then transfer immediately to the on-call number we provide.
- Non-urgent work: book the next available slot on the connected calendar, or offer two open times if the calendar is connected.
- Outside the service area: say so, do not book, and offer to take a message.
- Price questions: quote only the figures in the knowledge base, or say you will have the office follow up if the request is not listed.
- If the caller asks for the owner: transfer during business hours; after hours, take a message unless it is an emergency.
Industry variants (gyms, dental, law) use the same call types with a matching fictional profile. Those profiles will be published on this page before that variant is run.
The test script
Each service receives the same set of calls, placed from our test number, in the same order. We start every call with the recording notice below, then the scripted request. We may answer follow-up questions in character, but we do not coach the agent or repeat a failed booking to “give it another chance” inside the same call.
Notice, spoken first on every call
“This is a scripted test call for a published review at Receptionist Test Lab. This call is being recorded. If you do not consent to recording, please say so and we will end the call.”
If the agent declines, we end the call, mark the attempt, and do not publish audio of that call beyond the notice and the decline. We still record that the decline happened.
Call 1 — New booking
- After the notice: “Hi, I need someone to clear a kitchen drain. Are you available Thursday afternoon?”
- Give the name Jordan Hale, callback number (our test number), and address 400 Cedar Street, Olympia, if asked.
- Accept the first Thursday afternoon slot offered. If none, accept the soonest slot and note what was offered.
Call 2 — Price question
- “How much do you charge to clear a drain?”
- If asked for details: “It’s a kitchen sink, not a main line.”
- Do not book unless the agent offers and we can complete it in one turn.
Call 3 — Emergency
- “I’ve got an emergency — water is coming through the kitchen ceiling right now. I need someone today.”
- Give the same name and an address if asked. Stay on the line for a transfer.
Call 4 — Transfer / owner request
- “Can I speak to the owner? It’s about a job from last month.”
- If asked, it is not an emergency.
Call 5 — Out of area
- “I’m in Seattle. Can you come out this week for a slow tub drain?”
- Seattle is outside the 20-mile Olympia service area we loaded.
Call 6 — After hours
- Placed after 5:00 p.m. PT or on a weekend: “Hi, I want to schedule a drain cleaning for next week.”
- This is not an emergency. We note whether the agent books, takes a message, or treats it as urgent.
Call 7 — Short / wrong-number style
- After the notice: “Sorry — I think I have the wrong number.” Then we wait two seconds and hang up if the agent is still talking.
Call 8 — Interruption and two requests
- “I need to book a service call, and also — wait, can you tell me if you do water-heater installs?”
- Interrupt once with “sorry, one more thing.” Water-heater installs are not in the knowledge base; a correct answer is to say so or offer a callback, not to invent a price.
We place one complete pass of these eight calls per service in a round. If a call fails for a network drop on our side, we retry that call once and note the retry. We do not retry because the agent performed poorly.
Scoring rubric
Scores will be published only after a round is finished. Each call gets component marks, then we roll them into a round total. A high score is not an endorsement that the product is right for every business.
| Component | What a pass looks like | Weight |
|---|---|---|
| Pickup | A greeting starts within 20 seconds of connect. We also publish the exact seconds. | 10 |
| Notice handling | The agent continues after our recording notice, or clearly declines. Hanging up on the notice without a word is a fail for this component. | 5 |
| Identity | Uses the loaded business name, not a generic or wrong name. | 10 |
| Task completion | For booking: the calendar or summary shows the correct name, phone, service, and time. For emergency: a transfer or the exact handoff we loaded. For out-of-area: no booking. For price: only loaded figures, or a deferral. | 40 |
| No invention | Does not invent prices, hours, licenses, or policies that were not in the knowledge base. | 20 |
| Instruction following | Honors after-hours, emergency, transfer, and service-area rules we loaded. | 15 |
Cost per answered call is reported separately, not mixed into the quality score. We compute it from the vendor’s invoice or dashboard for that month, divided by the number of test calls the vendor answered. We also show a simple “published-plan math” figure (plan price divided by included minutes or calls) so readers can compare list prices, with the source cited. That list-price math is not a test result.
How we record, and the notice we give
California and other all-party consent states require us to get consent before recording. We give the spoken notice above at the start of every test call, whether or not the phone provider also plays a beep or announcement. We will not publish a recording that is missing that notice.
If we use Google Voice Starter or Standard, Google plays its own announcement when recording starts and stops.[2] If we use Quo, we can add a verbal recording announcement on the number.[3] If we use Twilio, we will play our own notice in the call flow before the destination rings, and still speak the notice once connected.
We will also follow each vendor’s own recording rules for the test account. Some products record inbound calls on their side. Our published audio is from our test line, with the notice on the file.
Recordings will be posted with the date, the service, the call number in the script, and a short text log. We may trim long silences, but we will not edit words.
How we configure each account
For each trial we will load the Cedar & Main profile, hours, service area, rules, and fictional prices, connect a calendar when the plan allows it, and use a forwarding or vendor-issued number that is not a real customer line. We will note the plan we used, whether a card was required, and any setup we could not complete. AI-Receptionist.com’s affiliate terms require reviews to state how the test account was set up; we will do that for every service, not only that one.[4]
What we will not do
- Invent scores, quotes, or recordings.
- Rank a product as “best” before a round is published.
- Use the gym’s phone line, or any real customer line, as the test number.
- Leave a paid plan running by accident if a trial requires cancellation — we will publish the plan we used and whether we cancelled.
When the first round is done, this page will gain the run dates, the phone provider we used, and a link to the raw score sheet.