← Home

Conversational AI Voice Agents

Forward-deployed across four production LLM voice-agent launches — solar, real estate, fitness and restaurant ordering — covering campaign setup, CRM integration, UAT and post-launch reliability.

Client work · 2026 · Forward Deployed Engineer · NuPlay AI

Stack: Python, FastAPI, AWS Lambda, Conductor, Soniox, Cartesia, Langfuse, Metabase

System map
Live call
Caller audiostreaming
Sonioxspeech to text
Turn controllerbarge-in · endpointing
LLMdialogue + tools
Cartesiatext to speech
Caller audiostreaming
In call
Conversation state
CRM lookup
Availability
Disposition
After call
AWS Lambdaevent driven
Transcript + summary
CRM / webhookidempotent
Metabase + Langfuse
The live call is latency-bound; the integrations, reporting and post-call work are allowed to be thorough.

Four production launches across solar, real estate, fitness and restaurant ordering. Each campaign started as a client workflow and ended as a live agent handling thousands of calls: prompt and tool design, CRM and webhook integration, UAT, a controlled release, and the reliability work that starts after go-live.

Volumes ranged from 2K to 40K leads. The common thread was not model quality but release discipline — a backup of the working configuration, a draft tested against real call cases, a publish step, and a live verification call before traffic moved.

Restaurant ordering: move the menu into the prompt

The original flow fetched the menu on every order. That cost latency and produced speech-to-text errors on dish names. I built a serverless AWS Lambda API that updates the menu section of the live prompt on demand, so the menu is already in context when the call begins. LLM dish understanding replaced similarity matching, which finally handled native Marathi and South Indian dishes.

Solar: one agent, two regions

The solar campaign confirmed visits after an initial call. I launched the campaign and a second region on a shared agent using Conductor-based workflow automation: an LLM step cleans addresses, and a post-call branch routes by city. The campaign produced 540 visits from 2K leads and 3K call attempts — 27% of leads.

Real estate: releases under live traffic

Real estate ran at 20K leads and 65K call attempts. Client-requested prompt releases covered pricing, call flow and language guards, each verified by diff and a re-fetch of the live configuration. An available-now callback added a post-call flag that feeds an email-alert workflow.

Fitness: scale and answer rate

Fitness pushed 40K leads with 56% answered. The tested STT, TTS, LLM and prompt configuration was promoted to production against a checklist with post-change verification. Voice selection and call-screener handling were tuned from real recordings rather than from the demo script.

A voice launch is not finished when the agent can talk. It is finished when the configuration can be changed safely while calls are running.

Seeing the system

Clients needed to see calls, leads and visit confirmations without reading transcripts. Metabase and SQL dashboards gave them those metrics; Langfuse traces gave me the root cause when a live agent behaved differently from the draft. Tracing turned vague reports — “it sounded confused” — into a specific turn, tool response or prompt change.

What I carry forward

Forward-deployed work is mostly boundary work: between the client's process and the agent, between a prompt change and a live campaign, between what the transcript says and what the CRM records. The model is one component in that contract. Release discipline and observability are what make the rest dependable.