Tool Retry & Backoff Plans

Overview

A backoffPlan configures retry logic for a tool call, so a transient failure doesn't need to end the conversation. It's available on hook-triggered tool calls and directly on apiRequest-type tools.

Fixed backoff uses a consistent delay between each retry attempt — for example, a two-second wait before every retry.

Exponential backoff increases the delay with each subsequent retry, for example 1s, then 2s, then 4s, then 8s. This spreads retries out further as failures continue, which is useful against an endpoint that's briefly overloaded rather than fully down.

You can exclude specific HTTP status codes from triggering a retry at all. This matters because not every failure is worth retrying — a 400 Bad Request, for example, means the request itself was malformed, and retrying it will just fail the same way every time. Reserve retries for status codes that indicate a transient condition, and exclude the ones that indicate a request-level problem.

{
  "type": "apiRequest",
  "function": {
    "name": "api_request_tool"
  },
  "name": "checkOrderStatus",
  "url": "https://api.yourcompany.com/orders/{{orderNumber}}",
  "method": "GET",
  "body": {
    "type": "object",
    "properties": {
      "orderNumber": {
        "description": "The user's order number",
        "type": "string"
      }
    },
    "required": ["orderNumber"]
  },
  "backoffPlan": {
    "type": "exponential",
    "maxRetries": 3,
    "baseDelaySeconds": 1
  },
  "timeoutSeconds": 45
}

If this API call fails, the platform retries up to three times with exponentially increasing delays starting at one second.

Retry configuration handles the mechanics of a failed call, but your system prompt should tell the assistant how to talk about it while retries are happening. Pair this with the error-recovery few-shot example described in Prompting Guide: Writing Effective Voice Prompts, which shows the assistant acknowledging a tool failure, retrying, and offering a transfer if retries are exhausted, for example:

Caller: Can you check my order status?
Assistant: One moment, let me look that up for you.
[tool call fails, retry in progress]
Assistant: That's taking a little longer than expected, but I'm still working on it.
[tool call fails again, retries exhausted]
Assistant: I'm having trouble reaching our order system right now. Let me transfer you to someone who can help directly.