Building a voice agent with ElevenLabs: the end-to-end tutorial

ElevenLabs can run a voice agent that answers the phone, replies, triggers actions and hands over to a human when it gets stuck. Configuration fits in half a day. The tuning that makes the agent genuinely usable then takes a few days of real calls and reviewed transcripts.

This tutorial walks through the six steps, from blank page to a ringing number. Two things to know up front. The vendor sells no phone numbers, so you have to bring one. And the advertised price covers only one of the three meters running during a call.

ElevenAgents, what the platform actually does

ElevenAgents is the name of ElevenLabs' conversational agent product, taken from the vendor's page in October 2026. The platform covers voice, written chat and email, in more than 90 languages. A voice agent there rests on four building blocks assembled by the vendor.

• Transcription. The in-house Scribe model listens to the caller and writes down what they say.
• Language model. You pick the model that does the reasoning, from the GPT family to Claude Opus 5.5, or you plug in your own.
• Speech synthesis. Eleven v4 Turbo landed in September 2026, advertised at around 100 milliseconds of latency.
• Turn taking. A proprietary model decides when the agent speaks and when it lets the caller speak.

This architecture chains three distinct technologies before producing sound. A direct audio model works differently, and I covered the gap between the two approaches in the article on the voice pipeline and voice-to-voice. The choice has a direct effect on perceived latency and on the bill.

Step 1, create the agent and say it is an AI

In the dashboard, create an agent from the blank template. You fill in a name, a language, then the first message. That first message is the sentence the agent speaks before anything else, and it is where your compliance with the European artificial intelligence regulation is decided.

Step 2, write a system prompt that holds up

The system prompt is the document that carries the agent's entire behaviour. A prompt written in a hurry produces a chatty agent that improvises and loses the caller. The structure I have used since my first deployments fits in six blocks.

1. The role. One sentence giving an identity and a tone, such as "you are the receptionist for company X".
2. The mission. What the agent must obtain: qualify a request, book an appointment, route to the right team.
3. The steps. The order in which it collects information: greet, ask for the first name, then the address.
4. Business context. Opening hours, response times, products, published prices.
5. Sample dialogue. Two or three realistic exchanges steer the style better than an abstract instruction.
6. A rules section. The hard constraints: short sentences, no listing out loud, rephrase when the caller is vague.

The single most profitable instruction concerns reply length. Cap it at one or two sentences. Spoken aloud, a four-line paragraph reads as a monologue and the caller cuts in. I detailed this prompt grid in my edition on what a voice agent can genuinely automate, and it has not changed since.

Step 3, load the knowledge base

The knowledge base section accepts documents and public pages. Once retrieval is switched on, the agent fetches the useful passage instead of carrying the whole catalogue in its prompt. This is the right home for frequent questions, terms and conditions and product sheets.

Keep the system prompt for behaviour and the knowledge base for facts. A prompt that swells to a thousand lines becomes impossible to maintain, and the agent starts mixing procedures up. Separating the two is what keeps an agent alive over time.

Step 4, give the agent tools

An agent without tools only talks. The platform ships eight ready-made system tools, which cover most of what a telephone switchboard needs.

• End the call, once the conversation is over.
• Detect the language, to switch to the caller's own.
• Transfer to another agent, specialised on a given topic.
• Transfer to a number, meaning to a human.
• Skip its turn, to wait without speaking.
• Play a keypad tone, to navigate a third-party phone menu.
• Detect voicemail, and leave a different message if needed.
• Update a variable, without calling any external interface.

Beyond those, you wire up your own actions. A tool calls an external interface to send a text message, read a calendar or write into the customer relationship manager. Since September 2026 the agent can fire several tools in the same turn, a setting now enabled by default.

Step 5, connect a phone number

This is the step everyone underestimates. ElevenLabs does not sell numbers. Two routes exist, and the right one depends on your volume.

• Import a Twilio number. You paste an identifier and an authentication token into the numbers tab, then pick the agent that takes the calls. A purchased number handles inbound and outbound. A verified caller identity only serves outbound.
• Connect your own carrier. A SIP protocol connection wires in your existing telephony infrastructure. The vendor exposes servers on fixed addresses, with entry points isolated per region, Europe included.

Allow time to obtain a French geographic number. The carrier asks for a company registration document, and validation takes a few days. Plan that step before promising a go-live date.

Step 6, outbound calls, and the French rule that changed

The platform can launch batched call campaigns, with a concurrency ceiling you set yourself. Voicemail detection avoids burning minutes on empty lines. Technically, nothing stops you firing thousands of calls from a spreadsheet.

The rule targets consumers, not inbound calls nor an existing customer relationship. The official detail is published by the French state services. The use cases that stay solid are the switchboard, appointment booking, calling back a prospect who just filled in a form, and following up a customer on an open case.

How much does an ElevenLabs voice agent cost?

An agent minute costs roughly 7 cents beyond the plan allowance, and that rate is identical from the free tier to the Business tier. The tier you pick therefore buys concurrency above all, meaning how many calls the agent can hold at once.

Two traps hide behind that table. Going over your concurrency ceiling is billed at double the normal minute, which gets expensive on a badly anticipated call peak. And language model tokens are paid on top, deducted from your credits, as the pricing page states. The phone line forms the third meter.

What blocks a European deployment

Where the data lives is a question to settle before the first minute of call time. ElevenLabs offers an isolated environment in Europe, but its documentation reserves it for Enterprise contracts. The self-service tiers give no access to it, and that is the point that sinks most projects in a regulated sector.

Two limits deserve reading before signing. Even inside the isolated environment, a custom language model or an end-of-call webhook can trigger processing outside the region. And zero retention mode, which prevents content being kept on the vendor's servers, is switched on separately. Put both questions to your sales contact, in writing.

A call recording contains a voice, therefore personal data. Tandem handles that point during scoping, before the first line of prompt. If this applies to you, our autopilot offer covers that scoping alongside the technical build.

Should you build this agent or buy a vertical solution?

That is the real question before starting. I put it publicly in a post on the build versus buy trade-off in July 2026, and the answer is not universal. An off-the-shelf solution deploys in days, with predictable costs and continuous updates. An in-house build takes months and a dedicated team.

Building your agent on ElevenLabs sits between the two. You keep control of the prompt, the tools and the voice, without writing any telephony infrastructure. If you are still weighing the platform, the voice agent platform comparison covers that choice in detail, and the voice agent guide covers the deployment method.

One last trade-off is worth raising early. Many requests that arrive labelled "voice agent" are better served in writing, and for far less money. I told one such story in the article on a tradesman's WhatsApp agent. The phone keeps its value when the caller already dials, and nobody picks up.

Should you build your voice agent with ElevenLabs?

Yes, for a switchboard, appointment booking or simple qualification, provided you accept three things. The voice is the best on the market and that is what you pay for, as I wrote in the audio generator comparison. The phone line stays your responsibility. And European hosting is negotiated in an Enterprise contract.

No, if your case requires data that never leaves Europe without an Enterprise budget, or if your need is an outbound campaign towards consumers. In that second case, French law has decided for you. For everything else, what I wrote about the quality shift in voice agents still holds, and the technology is no longer the limiting factor.