Custom LLM (Bring Your Own Model Endpoint)

Overview

Sulus supports using any OpenAI-compatible endpoint as your assistant's model, whether that's a self-hosted server, a fine-tuned OpenAI model, or a third-party LLM provider. This gives you full flexibility to bring your own model infrastructure instead of relying solely on a built-in provider.

Your server must expose an endpoint compatible with the OpenAI Chat Completions API. For best results, it should also support streaming completions.

Step 1 (optional): If your server requires authentication, register credentials via the /credential endpoint:

{
  "provider": "custom-llm",
  "apiKey": "<YOUR SERVER API KEY>"
}

If your server is not authenticated, you can skip this step.

Step 2: Create an assistant with the custom-llm provider:

{
  "name": "My Assistant",
  "model": {
    "provider": "custom-llm",
    "url": "<YOUR OPENAI COMPATIBLE ENDPOINT BASE URL>",
    "model": "my-cool-model",
    "messages": [
      {
        "role": "system",
        "content": "You are an assistant."
      }
    ],
    "temperature": 0.7
  }
}

Sulus supports two authentication methods for securing your custom LLM endpoint:

• API Key — Sulus sends a static API key in the Authorization header, which your server validates before processing requests
• OAuth2 Credentials — uses the OAuth2 client credentials flow with automatic token refresh; you configure an OAuth2 URL, client ID, and client secret in the Sulus dashboard

To use a fine-tuned OpenAI model specifically, follow these steps:

1. Set the custom LLM URL to https://api.openai.com/v1
2. Assign the custom LLM key to your OpenAI API key
3. Set the model field to your fine-tuned model ID (e.g. ft:gpt-4o-mini:org:name:id)
4. Execute a PATCH request to the /assistant endpoint and ensure model.metadataSendMode is set to off

Sulus supports providers like OpenRouter, AnyScale, Together AI, and others through the same OpenAI-compatible pattern.

Step 1: Register your API key via the /credential endpoint:

{
  "provider": "openrouter",
  "apiKey": "<YOUR OPENROUTER KEY>"
}

Step 2: Create an assistant using that provider:

{
  "name": "My Assistant",
  "model": {
    "provider": "openrouter",
    "model": "cognitivecomputations/dolphin-mixtral-8x7b",
    "messages": [
      {
        "role": "system",
        "content": "You are an assistant."
      }
    ],
    "temperature": 0.7
  }
}

The full list of providers that support bring-your-own-key for LLMs includes:

• OpenAI
• Anthropic
• Azure OpenAI
• Google Gemini
• Groq
• DeepSeek
• OpenRouter
• Together AI
• Cerebras
• DeepInfra
• Perplexity
• Anyscale
• xAI

Add provider API keys directly in the Sulus dashboard under the Integrations tab. Once validated, you won't be billed by Sulus for that provider's usage — you'll be billed directly by the provider instead.

In summary, whether you're pointing to your own server, a fine-tuned OpenAI model, or a third-party LLM provider, the pattern is the same: register credentials via /credential (or the dashboard), then configure your assistant's model field to reference that provider and model.