Retry Transient Model Calls

Providers fail now and then with rate limits and 503s. Let's make an agent route retry those, give a flaky tool its own retry loop, and turn retry off where failing fast matters:

import { agent } from "@b4run/sdk"
 
export default agent({
  model: "gpt-5-mini",
  retry: { maxAttempts: 5, baseDelay: 500 },
  systemPrompt: "Summarize the document the user provides in three bullet points.",
})

NotesCopy link to section: Notes

Retry is per route. Each agent() declares its own retry, so different routes can have different policies.
Each model call retries on its own. maxAttempts is the attempts per model call, not per run. A failure in a later model call of the tool loop sends only that call again; tools that already ran don't run again.
Only temporary failures retry. Server errors (5xx), network errors, timeouts and rate limits are retried. A quota 429, an invalid API key, a missing model and a malformed request fail immediately.
baseDelay paces capacity rate limits. A 429 with no Retry-After waits min(baseDelay * 2^n + jitter, 10s) before the next attempt. A 429 whose Retry-After is 60 seconds or less is waited out by LangChain; one over 60 seconds isn't retried, and the run fails at once. LangChain paces the other retries with its own backoff, starting at 1 to 2 seconds.
maxAttempts: 1 disables retry. If you omit retry, the default is 3.
A response that fails partway isn't retried. Retries happen before the model's response starts, so no token is ever sent twice. Once part of a response has streamed, an error comes through the stream.
Tool retry lives in the tool. Agent retry covers model and provider calls. For a tool, write the loop yourself, as fetch-doc.ts does above.