Multilingual Assistants

Overview

Sulus supports building multilingual voice assistants with automatic language detection and seamless language switching. Configuring this well requires three components working together: a multilingual transcriber, a multilingual voice, and a system prompt that explicitly instructs the assistant on which languages it supports.

The first step is configuring your transcriber to automatically detect and process multiple languages. Three providers support this:

Example using Deepgram (recommended for best speed/accuracy balance):

curl -X POST "https://api.sulus.ai/assistant" \
  -H "Authorization: Bearer $CORE_SYSTEM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Multilingual Assistant",
    "transcriber": {
      "provider": "deepgram",
      "model": "nova-2",
      "language": "multi"
    }
  }'

Configure your assistant to respond with the appropriate voice for each detected language. Azure is recommended for the broadest coverage (400+ voices across 140+ languages):

// Option 1: Automatic voice selection (recommended)
{
  "provider": "azure",
  "voiceId": "multilingual-auto"
}

// Option 2: Specific voices with a fallback list
{
  "provider": "azure",
  "voiceId": "en-US-AriaNeural",
  "fallbackPlan": {
    "voices": [
      { "provider": "azure", "voiceId": "es-ES-ElviraNeural" },
      { "provider": "azure", "voiceId": "fr-FR-DeniseNeural" },
      { "provider": "azure", "voiceId": "de-DE-KatjaNeural" }
    ]
  }
}

Important: Option 2's fallbackPlan is not a per-language voice switch keyed to the detected language — a fallback voice only engages if the primary voice provider fails mid-call. If your primary voice never fails, the fallback voices in that list are never used, regardless of what language the caller is speaking. For a voice that genuinely adapts to the detected language, use Option 1's multilingual-auto voice ID instead. See Per-Language Voice Selection Approaches for a full comparison of the real options here, and Voice Fallback Plans for the full fallbackPlan reference.

This is critical — you must explicitly list supported languages in your system prompt. Without this, assistants struggle to understand they can switch languages:

You are a helpful assistant that can communicate in English, Spanish, and French.

Language Instructions:
- You can speak and understand: English, Spanish, and French
- Automatically detect and respond in the user's language
- Switch languages seamlessly when the user changes languages
- Maintain consistent personality across all languages
- Use culturally appropriate greetings and formality levels

If a user speaks a language other than English, Spanish, or French, politely explain
that you only support these three languages and ask them to continue in one of them.

You can also add cultural tone guidance per language, for example:

• English: Direct, friendly, professional; solution-focused
• Spanish: Warm, respectful; use formal usted initially, then adapt
• French: Polite, courteous; use proper greeting conventions (Bonjour/Bonsoir)

Set a first message that signals support for multiple languages from the start:

"firstMessage": "Hello! I can assist you in English, Spanish, or French. How can I help you today?"

Or use language-specific greeting contents:

{
  "contents": [
    { "type": "text", "text": "Hello! How can I help you today?", "language": "en" },
    { "type": "text", "text": "¡Hola! ¿Cómo puedo ayudarte hoy?", "language": "es" },
    { "type": "text", "text": "Bonjour! Comment puis-je vous aider?", "language": "fr" }
  ]
}

The system prompt should explicitly define how the assistant handles language switching:

• Auto-detect: Automatically respond in the customer's language
• Language Switching: If the customer switches languages, switch with them seamlessly
• Mixed Languages: If the customer uses multiple languages, respond in their primary language
• Unsupported Languages: Politely explain which languages are supported

In summary, building a multilingual assistant on Sulus requires configuring three components together: a multilingual transcriber (Deepgram, Google, or Gladia), a multilingual voice provider (Azure recommended), and a system prompt that explicitly instructs the assistant to detect and switch languages. The system prompt instruction is especially critical — without it, language switching will not work reliably.