Model Config

Overview

Sulus allows you to configure the LLM (model) provider for your assistant in several ways, including using built-in providers, OpenAI-compatible endpoints, fine-tuned models, or your own custom server.

Sulus supports any OpenAI-compatible endpoint as the LLM, including services like OpenRouter, AnyScale, Together AI, or your own server. You can bring your own API keys for these providers via the Dashboard under the Integrations tab.

Step 1: Add your API key

POST your provider API key to the /credential endpoint:

{
  "provider": "openrouter",
  "apiKey": "<YOUR OPENROUTER KEY>"
}

Step 2: Create an assistant with the model provider

Specify the provider and model in the model parameter of your assistant configuration:

{
  "name": "My Assistant",
  "model": {
    "provider": "openrouter",
    "model": "cognitivecomputations/dolphin-mixtral-8x7b",
    "messages": [
      {
        "role": "system",
        "content": "You are an assistant."
      }
    ],
    "temperature": 0.7
  }
}

To use a fine-tuned OpenAI model, follow these steps:

1. Set the custom LLM URL to https://api.openai.com/v1
2. Assign the custom LLM key to your OpenAI key
3. Update the model field to your fine-tuned model ID
4. Execute a PATCH request to the /assistant endpoint and ensure model.metadataSendMode is set to off

You can also use your own OpenAI-compatible server as the LLM. If your server requires authentication, POST your server's API key via the /credential endpoint:

{
  "provider": "custom-llm",
  "apiKey": "<YOUR SERVER API KEY>"
}

Then create an assistant using the custom-llm provider:

{
  "name": "My Assistant",
  "model": {
    "provider": "custom-llm",
    "url": "<YOUR OPENAI COMPATIBLE ENDPOINT BASE URL>",
    "model": "my-cool-model",
    "messages": [
      {
        "role": "system",
        "content": "You are an assistant."
      }
    ],
    "temperature": 0.7
  }
}

For best results, your custom server endpoint should support streaming completions.

If your organization has HIPAA mode enabled, only the following LLM providers are permitted:

• OpenAI
• Azure OpenAI
• Anthropic
• Anthropic Bedrock / Vertex
• Google
• Custom LLM
• Baseten
• Together AI
• xAI

Model cost is heavily influenced by your assistant's configuration. Longer system prompts and more tool definitions increase token usage and therefore cost. Using models that support prompt caching and keeping prompts concise are effective ways to reduce LLM costs.

In summary, Sulus gives you flexible options for LLM configuration — from managed providers to fully custom servers — all configured through the model parameter of your assistant. Once your API key is validated via the Dashboard or /credential endpoint, you'll be billed directly by the provider rather than through Sulus.