Groq & OpenAI-Compatible Endpoints

Overview

Beyond the built-in model providers covered in Model Config, Sulus supports Groq as a dedicated low-latency model provider, plus a broader umbrella of "any OpenAI-compatible endpoint" support that covers your own server or a range of third-party inference services. This page covers both. For a faster way to get started without picking individual providers, see Model Presets; for the general bring-your-own-endpoint pattern in more depth, see Custom LLM (Bring Your Own Model Endpoint).

Groq is a model provider built around specialized inference hardware, optimized specifically for rapid AI processing. Its main advantages are:

• Low latency — minimal delay generating a response, which matters because the model-response stage is typically the largest component of total voice-to-voice latency in a call
• High throughput — capable of handling large volumes of requests efficiently
• Scalability — scales cleanly as call volume grows

Groq is best suited to use cases where speed is the top priority, such as high-volume, relatively straightforward conversations where an instant-feeling reply matters more than deep reasoning. As with any fast, lightweight model, test your assistant in real conversations to confirm reasoning quality is still acceptable for your use case — the fastest models trade some capability for speed.

To use Groq, set provider to "groq" in your assistant's model configuration, either in the dashboard or via the API:

{
  "model": {
    "provider": "groq",
    "model": "llama-3.3-70b-versatile",
    "messages": [
      {
        "role": "system",
        "content": "You are an assistant."
      }
    ]
  }
}

Check the dashboard's model picker for the current list of supported Groq models — available models can change as Groq updates its lineup.

Separately from named providers like Groq, Sulus supports pointing your assistant's model at any endpoint that implements the OpenAI Chat Completions API. This is the same custom-llm pattern documented in full on the Custom LLM (Bring Your Own Model Endpoint) page, and it covers:

• A server you host yourself
• A fine-tuned OpenAI model
• Third-party OpenAI-compatible inference services such as OpenRouter, AnyScale, or Together AI

Other services that expose an OpenAI-compatible Chat Completions API can typically be used the same way, even if they aren't named explicitly here — the requirement is just that the endpoint speaks the OpenAI Chat Completions format (streaming support is recommended for best results).

For Groq and for supported third-party LLM providers, you can add your own API key directly in the Sulus dashboard under the Integrations tab. Once validated, you won't be billed through Sulus for that provider's usage — you'll be billed directly by the provider instead.

This is the same bring-your-own-key pattern used across Sulus for models, transcribers, and voices — see Provider Keys (BYOK) for the general reference.

If your organization has HIPAA mode enabled, be aware that Groq and generic custom OpenAI-compatible endpoints outside Sulus's approved subprocessor list are not HIPAA-compliant options for the model layer. Under HIPAA mode, only the approved LLM providers listed on HIPAA Subprocessors & Data Residency (which does include the general-purpose custom-llm provider itself, pointed at your own HIPAA-compliant server) may be used. Configuring a non-compliant provider while HIPAA mode is active will be rejected.

In summary: Groq and the OpenAI-compatible endpoint pattern give you real flexibility to optimize for latency or bring your own inference stack, but if you operate under HIPAA mode, confirm the specific provider or server you're pointing to is on the approved list before switching to it.