Model Fallback

Overview

The model.fallbackModels array lets you configure backup LLM providers that activate automatically if your assistant's primary model fails, keeping the call running instead of ending in an error. This mirrors the same fallback pattern used for voice and transcription — see Voice Fallback Plans and Transcriber Config for the equivalent mechanisms in those layers.

When the primary model fails, Sulus tries each entry in fallbackModels sequentially, in the order you list them, until one succeeds. If every fallback model is exhausted without a success, the call ends with an LLM-related error — see Ended Reason: Complete Reference for the exact endedReason codes this can produce.

Each entry in fallbackModels is a full model configuration object, just like the primary model:

{
  "model": {
    "provider": "openai",
    "model": "gpt-4o",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant."
      }
    ],
    "fallbackModels": [
      {
        "provider": "anthropic",
        "model": "claude-3-5-sonnet-20241022"
      },
      {
        "provider": "google",
        "model": "gemini-1.5-pro"
      }
    ]
  }
}

In this example, if the primary OpenAI model fails, Sulus tries the Anthropic model first, then the Google model, before giving up.

• Rate limit or quota exceeded on the primary provider
• Provider server errors (5xx-style failures)
• Generic LLM failures reported by the pipeline

See Ended Reason: Complete Reference for the exact codes associated with each of these (they follow an HTTP-status-based naming pattern, e.g. *-429-rate-limit-reached, *-500-server-error, *-llm-failed).

If your organization has HIPAA mode enabled, every provider listed in fallbackModels must also be HIPAA-compliant — a non-compliant fallback provider is not exempt just because it's a backup rather than the primary model. See HIPAA Subprocessors & Data Residency for the current approved LLM provider list.

• Use genuinely different providers across your fallback list rather than just different models from the same provider — a provider-wide outage takes down every model on that provider at once.
• Order fallbacks by preference or cost, since the first entry is tried first.
• Confirm every fallback model supports the same capabilities your assistant depends on, such as function calling or structured outputs.
• Pair LLM fallback with a voice fallback plan and a transcriber fallback plan for full pipeline redundancy — a resilient model layer doesn't help if the voice or transcription layer has no backup of its own.

In summary, fallbackModels is a straightforward safety net for the model layer of your assistant: list backup providers in priority order, and Sulus works through them automatically if your primary model fails.