AI Receptionist (Voice Agent)

Problem

Every business that relies on the phone has the same leak: calls that come in while staff are busy, at lunch or gone for the day. WebCode Technology wanted a receptionist that picks up every time, greets the caller, answers questions about the company's services and passes the call to a person when the question needs one. Ready-made AI receptionists exist, but they don't know your services, your rules or your phone setup, and most are built English-first. WebCode's callers speak Bangla. Two things had to be right from the start. The agent couldn't make anything up, because a receptionist that invents a price or a service is worse than no receptionist. And it had to feel like a phone call, not a voice menu: quick replies, and silence the moment the caller starts talking.

Approach

The agent uses a speech-to-speech model that hears the caller's audio and speaks back directly, instead of the usual chain of speech-to-text, a language model and text-to-speech. That chain adds 2 to 3 seconds to every turn, which is what makes most phone bots feel slow. It started on the OpenAI Realtime API, which accepts phone audio (G.711 μ-law) as it is. Since 14 October it runs on Google Gemini Live: the agent converts the caller's audio for it, and filters and downsamples Gemini's reply back to phone audio. Both providers sit behind the same session interface, and the OpenAI version stays as a fallback. The first version ran on Twilio because Twilio works in minutes. All the phone code sits behind one small interface, one part for the audio and one for call control, and the AI code never imports Twilio. When the company moved to its own SIP line, an Asterisk adapter plugged into the same interface. The agent learns the business from a knowledge file built from the company website, and it may state only those facts. If a caller asks for a price that isn't in the file, it says so and gives the office number. Deployment follows the company's existing setup: a push to main builds a Docker image on a self-hosted GitHub Actions runner and restarts the containers behind nginx with TLS.

Key decisions

• Test the cheaper model on real calls before choosing it

  We switched to gpt-realtime-2.1-mini to save money and tested it on live calls. It narrated odd filler lines and handled the language noticeably worse, so we went back to the full model the same day. Audio minutes are the biggest running cost of an AI receptionist, which is exactly why the choice was made on real calls and not on the price list.

• Switch AI providers when the language and the price both say so

  On OpenAI Realtime the Bangla voice sounded Indian rather than Bangladeshi, and on one recorded call the transcription came back in Hindi script. It was also the biggest running cost, about $0.17 a call minute. Google Gemini Live can be set to Bangladeshi Bangla, and in early measured calls it cost about $0.03 a minute, roughly 5 times less. Because the AI already sat behind one interface, the switch changed the provider, not the call handling.

• Fewer tools, not more

  Early test calls went wrong in ways that had nothing to do with speech. The agent invented a list of services, kept asking callers for an account number because we'd given it a customer lookup tool, and pushed support tickets. Removing the lookup and ticket tools fixed most of it. In the test phase it has two: end the call, and transfer to a human. Calls became short and on topic. Integrations come back one at a time, once the basics hold. The first was booking requests: the agent takes the details and a reference number, and the clinic confirms the time by SMS.

• Protect the greeting from line noise

  Noise at the start of a call was heard as speech (once as the word 'Nagasaki') and restarted the greeting three times. Now listening stays off until the greeting has fully played, then the input is cleared and turn detection starts, with an 800 ms silence window so callers who pause mid-sentence aren't cut off. The trade-off is stated plainly: anything said during the six-second greeting is ignored.

• Software instead of a phone system box

  We priced a hardware phone system (about USD 350) before asking what it actually had to do: receive calls from the SIP trunk and hand them to the AI. Asterisk does that as free software in a Docker container next to the agent, deployed with the same pipeline. A box in the office would also stop taking calls whenever the office lost power or internet.

• Follow the local phone rules, then the code stays the same

  Telecom rules differ by country. Here, only licensed providers may sell SIP trunks, and the provider wouldn't connect one to a server abroad, so the phone system and the agent moved to a VPS in the same country as the callers. Because of the interface from day one, that was a change of where the code runs, not of the code. The same applies to a US, UK or Australian carrier.

A hard problem I solved

The hardest bug was an agent that wouldn't stop talking. Callers said "stop" and it carried on, which is the fastest way to lose a caller. The cause was timing, not the AI: the model generates a reply in about a second, but the caller hears it over 5 to 10 seconds. The code marked the agent as finished speaking when generation finished, so an interruption a few seconds later found nothing to stop, while the phone kept playing queued audio. The fix tracks what the caller has heard, not what the model has produced. After each chunk of audio the agent sends a marker, and the phone side sends it back once that audio has actually played. While markers are outstanding, the agent is still speaking. When the caller interrupts, the phone's queue is cleared and the model's message is cut to the milliseconds the caller really heard, so the AI doesn't believe it said things nobody heard. Asterisk has no built-in markers, so its adapter sends audio at real-time pace in 20 ms frames and fires the markers itself. In a simulated call with real speech, the agent went quiet 138 ms after the caller said "stop" and answered only "Yes, go ahead."

Status

In testing with WebCode Technology's own test and fix team since 24 October 2026, answering calls on local business numbers in Bangladesh, including a dental chamber's line. The Twilio version answered calls on 7 October, the first day of the project. The SIP path, about 250 lines of adapter code, was deployed on 12 October with no changes to the AI code. Since then it has moved to Gemini Live, learned the dental chamber's real answers from 135 recorded calls, started taking booking requests confirmed by SMS, and gained a portal where each business sees its calls, recordings, costs and the questions it couldn't answer.