Docs Agent

Overview

This is a full worked walkthrough for building a voice assistant that answers caller questions about your own documentation using Retrieval-Augmented Generation (RAG). The assistant indexes your documentation in LlamaCloud (a separate document-indexing service), queries that index through a custom tool during the call, and uses the retrieved passages to answer accurately instead of relying on the model's own memory of your docs.

This is a different approach from the platform's built-in knowledge base tool covered in Query Tool Setup, which is backed by Trieve and manages indexing for you inside Sulus itself. Use this RAG walkthrough when you want to manage your own external index — for example if you already maintain a LlamaCloud project, need custom chunking or retrieval logic, or want to reuse the same index across tools outside of Sulus. Use the built-in Query Tool when you want the simplest path and don't need to manage indexing yourself.

What you'll build:

• A LlamaCloud index of your documentation
• A RAG query tool that retrieves from that index
• An assistant with the tool attached and a system prompt tuned for voice support
• (Optionally) an analysis plan to monitor call quality over time

Before building anything in Sulus, index your documentation as a searchable vector store in LlamaCloud:

1. Create a new project in LlamaCloud
2. Consolidate your documentation into a single file where possible (a combined text export works better for retrieval than many small files)
3. Upload the file and set the embedding model to the recommended default
4. Set chunking to 512 tokens with a 50 token overlap
5. Note your pipeline/index ID and API key — you'll need both in Step 2

Chunk size and overlap matter here: 512 tokens keeps each retrieved passage focused enough to be useful in a voice answer, and the 50-token overlap prevents an answer from being cut off right at a chunk boundary.

Create an apiRequest-type tool that queries your LlamaCloud index. The tool needs a name, the LlamaCloud retrieve endpoint as its URL, the POST method, and a query parameter templated from whatever the caller asks:

curl -X POST "https://api.sulus.ai/tool" \
  -H "Authorization: Bearer $CORE_SYSTEM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "apiRequest",
    "name": "docsquery",
    "function": {
      "name": "docsquery",
      "parameters": {
        "type": "object",
        "properties": {
          "query": {
            "type": "string",
            "description": "The search query to find relevant documentation"
          }
        },
        "required": ["query"]
      }
    },
    "url": "https://api.cloud.llamaindex.ai/api/v1/pipelines/YOUR_PIPELINE_ID/retrieve",
    "method": "POST",
    "headers": {
      "type": "object",
      "properties": {
        "Authorization": {
          "type": "string",
          "value": "Bearer YOUR_LLAMACLOUD_API_KEY"
        }
      }
    },
    "body": {
      "type": "object",
      "properties": {
        "query": { "type": "string", "value": "{{query}}" }
      }
    }
  }'

Save the returned tool id — you'll attach it to the assistant in Step 3. See Custom Function Tools for the general field reference this tool type shares with custom function tools.

Create the assistant with the RAG tool attached, using Anthropic's Claude model as the LLM provider and a first message that frames the assistant as a support agent:

import { CoreSystemClient } from "@core-system/server-sdk";

const client = new CoreSystemClient({ token: process.env.CORE_SYSTEM_API_KEY });

const assistant = await client.assistants.create({
  name: "Docs agent",
  firstMessage: "Hey, I'm a support agent here to help with documentation questions. What can I help you with?",
  model: {
    provider: "anthropic",
    model: "claude-3-5-sonnet-20241022",
    maxTokens: 400,
    messages: [
      {
        role: "system",
        content: `You are a helpful documentation support assistant. Always use the docsquery tool for specific documentation questions rather than answering from memory. Keep your responses conversational and voice-friendly -- short sentences, no bullet points or markdown. Summarize complex or technical information simply. If a caller's question is unclear or too broad, ask a clarifying question before querying.`
      }
    ],
    toolIds: [ragToolId]
  }
});

The system prompt above encodes four guidelines worth keeping regardless of your own wording: always use the query tool for specific documentation questions, keep responses conversational for voice, summarize complex information simply, and ask a clarifying question when the request is unclear.

To monitor how well the docs agent is answering over time, add an analysis plan when creating or updating the assistant:

analysisPlan: {
  summaryPlan: {
    enabled: true,
    prompt: "Summarize this documentation support call."
  },
  successEvaluationPlan: {
    enabled: true,
    prompt: "Did the caller get a helpful, accurate answer to their documentation question?",
    rubric: "NumericScale"
  }
}

See Call Analysis for the full set of summary and success-evaluation options, including the available rubric types.

• Custom Function Tools — the general field reference for the tool type used here to query LlamaCloud
• Query Tool Setup — the platform's own built-in, Trieve-backed knowledge base tool; use it instead of this walkthrough if you don't need to manage an external index yourself
• Call Analysis — full detail on summary and success-evaluation plans