Dynamic Multilingual Agent

Overview

This page is a full worked walkthrough of the single-assistant, automatic-language-detection approach to multilingual support: one assistant that listens for whatever language the caller is using, responds in kind, and switches languages mid-call if the caller does. This is a different approach from the Multilingual Support Squad Template, which has callers explicitly select a language up front and then routes them to a dedicated assistant built for that language — see the comparison table at the end of this page for when each is the better fit. This walkthrough builds on the concepts introduced in Multilingual Assistants, Transcriber Provider Comparison, Voice Fallback Plans, and Per-Language Voice Selection Approaches; it's a concrete worked application of those mechanisms rather than a new one.

The system prompt is what actually makes the assistant able to switch languages — without it explicitly naming its supported languages, giving per-language tone guidance, and stating the switching rule, an assistant will often default back to one language and won't reliably follow a caller who switches mid-call. Here's a worked example for a multilingual customer support assistant:

You are Maria, a multilingual customer support representative.
You help customers in English, Spanish, and French.

LANGUAGE CAPABILITIES:
- English: Primary language for North American customers
- Spanish: For customers in Spain, Mexico, and Latin America
- French: For customers in France, Canada, and francophone regions

CULTURAL GUIDELINES:
- English: Direct, friendly, professional tone
- Spanish: Warm and respectful; use the formal "usted" initially, then adapt to the caller's own register
- French: Polite and formal; use proper greeting conventions ("Bonjour"/"Bonsoir")

Always respond in whatever language the customer is currently using.
If the customer switches languages mid-call, switch with them seamlessly - do not ask them to confirm or restate anything.
If a caller speaks a language you don't support, politely explain which languages you do support and ask them to continue in one of those.

Notice the prompt does three distinct jobs: it names the exact supported languages, it gives per-language cultural and tone guidance so the register feels natural rather than merely translated, and it states the switching rule explicitly. All three matter — a prompt that only lists supported languages without the explicit switching instruction is the most common reason a "multilingual" assistant fails to actually follow a caller across languages.

Your transcriber needs to detect the spoken language in real time, independent of the system prompt. Deepgram is the primary recommendation, using the "multi" language setting on Nova 2 or Nova 3:

curl -X POST "https://api.sulus.ai/assistant" \
  -H "Authorization: Bearer $CORE_SYSTEM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "GlobalTech Support Agent",
    "transcriber": {
      "provider": "deepgram",
      "model": "nova-2",
      "language": "multi"
    }
  }'

Google's multilingual model is a reasonable alternative when you need its specific language coverage, using "language": "multilingual" on the latest model:

{
  "transcriber": {
    "provider": "google",
    "model": "latest",
    "language": "multilingual"
  }
}

See Transcriber Provider Comparison for a full feature-by-feature comparison of Deepgram against other multilingual-capable providers, including code-switching support and latency tradeoffs.

Pair your transcriber with a voice configuration that can respond appropriately across your supported languages. Set a primary voice, then list per-language voice IDs in a fallbackPlan:

{
  "voice": {
    "provider": "azure",
    "voiceId": "en-US-AriaNeural",
    "fallbackPlan": {
      "voices": [
        { "provider": "azure", "voiceId": "es-ES-ElviraNeural" },
        { "provider": "azure", "voiceId": "fr-FR-DeniseNeural" },
        { "provider": "azure", "voiceId": "es-MX-DaliaNeural" }
      ]
    }
  }
}

Important caveat: this is the same fallbackPlan mechanism documented on Voice Fallback Plans, and as Per-Language Voice Selection Approaches spells out, a fallback list only actually engages if the primary voice provider fails mid-call — it does not switch based on the language the caller is speaking. If your primary voice never fails, the fallback voices in this list are never used, regardless of what language the call is in. If you need a voice that truly adapts automatically to the detected language rather than a curated failover list, use Approach A or B from Per-Language Voice Selection Approaches instead — for example, Azure's multilingual-auto voice ID:

{
  "voice": {
    "provider": "azure",
    "voiceId": "multilingual-auto"
  }
}

Azure is recommended in either case for its breadth of coverage (400+ voices across 140+ languages).

Set a first message that signals multilingual support from the start of the call, so callers know they don't need to search for a menu option or a different phone number:

"firstMessage": "Hello! I can assist you in English, Spanish, or French. How can I help you today?"

Together, the language-aware system prompt, the multilingual transcriber, the per-language voice configuration, and this greeting are what make a single assistant behave as though it's fluent across languages — the transcriber detects what's being said, the system prompt tells the model to answer in kind, and the voice configuration determines what that answer sounds like.

Both approaches solve multilingual support, but they make different tradeoffs:

Choose the dynamic approach on this page when you want callers to speak naturally without an upfront menu, or expect callers to code-switch mid-conversation. Choose the Squad-based approach when maximal per-language prompt and voice tuning matters more than a seamless mid-call switch. See Multilingual Assistants for the underlying single-assistant configuration concepts this walkthrough applies, and Multilingual Support Squad Template for the full Squad routing flow.