Flight Planner Web UI

B4.run's navlog demo, a VFR flight planner, ships with a CopilotKit web client that talks to the server over AG-UI. Let's walk through how it's wired. The client is a workbench, not a chat widget. It renders its own thread rail, transcript, and composer in place of CopilotSidebar, so the streamed brief and the plan, subagent, tool, and approval cards all appear inline in message order.

This recipe covers the application wiring. For the endpoint, events, threading, and interrupt outcomes, see AG-UI and Web Clients.

What you'll build#Copy link to section: What you'll build

The demo lives in examples/navlog: a B4.run server (server/) and a Next.js CopilotKit client (web/). The client provides:

• Thread rail: "New conversation" plus the list of threads, each titled from its first user message.
• Chat + report: streamed markdown output and citations in the app's own transcript.
• Plan cards: the root checklist updates in place while the plan runs.
• Subagent cards: the weather and performance subagents' delegated-work status, child checklist progress, and a bounded trail of child-tool names and statuses.
• Suggestions + tools: three discovery prompts on the empty state, and generic root-tool cards in the transcript.
• Permissions: the standard interrupt UI owns the approve and deny actions, rendered at the end of the transcript where the run stopped.
• Composer: send, attach an image when the route's model takes one, and stop while a run is in flight. It's blocked while the agent is running or waiting on an approval, and the header says which.
• Media: images, audio, video, and documents in user messages and tool results, plus a notice for each part the model never saw.

The plan and subagent cards only display progress. Interrupts are resolved through the standard AG-UI fields described on the protocol page.

The Workbench also reviews durable memory. A small panel reaches B4.run through the same allowlisted, same-origin proxy used for thread hydration. See Approve memory candidates below.

Run it#Copy link to section: Run it

1. 1

   Configure the server

   bash

   cd examples/navlog/server
   cp .env.example .env   # set OPENAI_API_KEY here, not in the web app

2. 2

   Start both apps

   bash

   cd examples/navlog
   pnpm install
   pnpm dev          # B4.run server on :3002, web client on :3010

   Open http://localhost:3010 and choose Plan a flight.

Connect the client to B4.run#Copy link to section: Connect the client to B4.run

The Next.js runtime route registers a B4HttpAgent, an AG-UI HttpAgent that also reports the route's capabilities, pointed at the encoded B4.run assistant id. Register it under CopilotKit's default agent id so every hook binds without per-component wiring:

import { B4HttpAgent } from "@b4run/ag-ui/client"
import { CopilotRuntime, createCopilotRuntimeHandler } from "@copilotkit/runtime/v2"
 
export const runtime = "nodejs"
export const dynamic = "force-dynamic"
 
const b4Url = process.env.B4_SERVER_URL ?? "http://127.0.0.1:3002"
const agUiUrl = `${b4Url}/agui/${encodeURIComponent("/navlog#agent")}`
 
const handler = createCopilotRuntimeHandler({
  runtime: new CopilotRuntime({
    agents: { default: new B4HttpAgent({ url: agUiUrl }) },
  }),
  basePath: "/api/copilotkit",
})
 
export const GET = handler
export const POST = handler

The catch-all route is required. It exposes CopilotKit's V2 REST and SSE paths under /api/copilotkit/*. Setting useSingleEndpoint={false} makes the browser start with GET /api/copilotkit/info rather than send the legacy method envelope to the base URL. Only the B4.run server has OPENAI_API_KEY. The web runtime holds no model credential.

Three pieces of that tree matter most:

• defaultThrottleMs={100}: the re-render throttle defaults to unthrottled, and a full planning run streams hundreds of events, which pegs the renderer.
• CopilotChatConfigurationProvider: CopilotKit does not provide one (<CopilotChat> and <CopilotSidebar> did). Without it, every thread-aware hook falls back to the agent's own auto-minted thread, and selecting a row in the rail would change nothing.
• <DemoSuggestions /> and <ToolCallCard /> render nothing. They publish into CopilotKit's registries, and the transcript reads them back.

The workbench shell#Copy link to section: The workbench shell

AppShell is the only component that talks to the agent. It calls useAgent() with no arguments. That unscoped form takes its thread from the surrounding chat configuration, so the transcript, useInterrupt, useSuggestions, and the tool-call renderers all resolve the same agent and thread.

Dropping the sidebar has two consequences you'll want to handle if you build your own shell:

• Run failures need a subscription. copilotkit.runAgent does not reject when a run fails. It catches, emits an error, and resolves normally. Failures surface through copilotkit.subscribe({ onError }), which the sidebar used to do for you.
• The permission gate needs renderInChat: false. The default (true) publishes the element into <CopilotChat>/<CopilotSidebar>. With neither mounted, the gate renders nowhere: the run parks with no approve or deny UI, no error, and green tests. With it set, useInterrupt returns the element and Transcript places it at the end of the message list.

Threads are local to the browser. B4.run's server can create and fetch a thread by id, but it has no endpoint to list threads. So the rail keeps its own list in localStorage behind a ThreadSource interface (examples/navlog/web/app/lib/thread-source.ts), and the app skips CopilotKit's useThreads.

The same seam restores history when you switch threads. CopilotKit's own replay path (connectAgent) only runs inside <CopilotChat>, which this app leaves out. So the shell reads GET /threads/:id/state through the proxy and maps the checkpoint's LangChain envelopes into the shapes the transcript already renders (app/lib/hydrate.ts). Messages, tool calls and results, and the plan come back. Subagent activity cards from earlier runs aren't checkpointed, so they don't, and the app says so in a line above the restored messages.

The same seam also carries GET /threads/:id/pending_interrupts, which lets a permission prompt survive a reload. useInterrupt only gets its state from live run events, so after a reload the server is still holding the gate and nothing on screen says so. app/components/HydratedInterrupts.tsx asks for the parked interrupts and reports their count upward, so the composer stays blocked until one is answered.

To restyle the app, edit examples/navlog/web/app/theme.css. It defines the palette as CSS variables and re-exports it as Tailwind tokens through @theme inline. That's why the app's utilities read bg-wb-surface, border-wb-border, and rounded-wb.

Attach an image#Copy link to section: Attach an image

The attach control appears only when the route's model takes an image. AppShell reads that from the route's capability document:

const capabilities = useCapabilities()
const canAttachImages = capabilities?.multimodal?.input?.image === true

There is no browser call to make here. In the browser, the agent is CopilotKit's runtime proxy, and it carries what the runtime's /info sync fetched through B4HttpAgent.getCapabilities(). useCapabilities() returns undefined until that handshake lands, which keeps the control hidden until then.

The composer reads a picked file as an inline image part, with the bytes and mimeType split out of the data: URL and the filename in metadata. On send, the shell puts the text first and adds the message itself:

const content: string | B4ContentPart[] =
  parts.length > 0
    ? [...(text.length > 0 ? [{ type: "text" as const, text }] : []), ...parts]
    : text
agent.addMessage({ id: globalThis.crypto.randomUUID(), role: "user", content })

A text-only turn stays a plain string, so it's exactly what it was before attachments existed.

The composer accepts PNG, JPEG, GIF and WebP images, the types OpenAI's vision input takes, and refuses any file over 4 MB. The cap matters because every turn sends the whole conversation to /agui, whose request body limit is 8 MiB, so one large image would break every turn after it.

Media in the transcript#Copy link to section: Media in the transcript

User bubbles and tool cards draw their media with app/components/MediaParts.tsx: an <img> for an image, the native players for audio and video, a download chip for a document, and a chip naming the handle for a provider file source, which the browser can't load. A url source is loaded only when it's http(s), because tool results are model-driven.

Each part's MIME type has to match what the part claims to be. An image part whose type isn't image/* renders as a chip, and so does audio or video with the wrong type. Only PDF, CSV, Markdown and plain-text documents get a download link, and the downloaded file is named from the type (document.pdf), never from the part's filename, which is shown only as the label.

Tool-result media is drawn beside the card, not inside it. CopilotKit's tool renderer only receives contentToText(content), so the parts are flattened to text before ToolCallCard sees them. The app's Transcript keeps the result's parts and renders MediaParts under the card itself.

Parts the model didn't see#Copy link to section: Parts the model didn't see

When the model or its provider can't take a part, B4.run drops it and emits a CUSTOM event named b4.content_parts_dropped. AppShell subscribes with onCustomEvent and the transcript turns each event into a muted line, such as 1 content part was not sent to the model: image (tool_result_media_unsupported). A notice with a toolCallId follows that tool call. One without it is about the user's own message, so the shell stamps it with the newest user message's id and it stays after that turn.

CopilotKit keeps no record of CUSTOM events, so the notices live only in the shell's state. Switching threads clears them, and a reload or a restored thread doesn't bring them back.

Restored threads#Copy link to section: Restored threads

Hydration (app/lib/hydrate.ts) brings media back too. A checkpointed user message holds LangChain content blocks, not AG-UI parts, so blocksToParts in app/lib/parts.ts maps them back. A tool that returned parts keeps the full list in the tool message's additional_kwargs.b4_content_parts, while its content holds only what the model could take. The UI shows what the tool produced, so those parts win.

A user image the adapter dropped is a different case. It never reached the model, so it isn't in the checkpoint, and it doesn't come back when the thread is restored.

A chart the model doesn't see#Copy link to section: A chart the model doesn't see

The navlog server ships renderChart (server/src/tools/renderChart.ts). Given a title and up to 12 { label, value } pairs, it returns two parts: a one-line text summary and an inline SVG bar chart.

return [
  { type: "text", text: summary },
  { type: "image", source: { type: "data", value: base64(svg), mimeType: "image/svg+xml" } },
]

The user sees the chart under the tool card. The model, gpt-5-mini, sees only the summary, because media in a tool result never reaches an OpenAI model. So every renderChart call also produces a dropped-part notice. That's expected: the notice is telling you the truth about what the model saw.

Render plan and subagent activities#Copy link to section: Render plan and subagent activities

The plan and subagent cards ship with the adapter. Install the package and hand its renderer array to CopilotKit:

npm install @b4run/ag-ui

import { b4ActivityRenderers } from "@b4run/ag-ui/react"
 
<CopilotKit
  runtimeUrl="/api/copilotkit"
  useSingleEndpoint={false}
  renderActivityMessages={b4ActivityRenderers}
>

Then import the stylesheet once, in your root layout:

import "@b4run/ag-ui/react/styles.css"

The cards carry no inline styles, so without that import they render as bare markup. To restyle them, override the --b4-activity-* custom properties in your own CSS. The package README walks through every customization option.

b4ActivityRenderers is a module-scope constant, so the registry keeps a stable identity across React renders. Its renderer is keyed by the adapter's own activity-type constant and validates the activity content with a strict runtime schema, so it rejects a payload with unknown or incompatible fields rather than dump arbitrary JSON into the transcript.

The example passes its own workbenchActivityRenderers, from examples/navlog/web/app/components/activity-renderers.tsx. It wraps the packaged PlanActivityCard and passes the packaged content schema, adding only per-part classes through the classNames prop, so validation and bounds stay in the package, where they're tested.

Subagents are not activities. The transcript (app/components/Transcript.tsx) calls useSubagentRuns(agent) on the useAgent() instance and mounts the packaged SubagentPanel on the result: the hook subscribes to the agent's event stream and folds SUBAGENT_STARTED/FINISHED/ERROR plus every event tagged subagentRunId into a tree of invocations, and the panel renders them nested — name, status, reasoning, prose, plan, tool calls with arguments and results, and the final result or error. The same classNames rule applies to the panel.

There's one catch with classNames. An entry can only set a property that the package stylesheet leaves unset on that element, because the package's CSS is unlayered and Tailwind's utilities are layered. For the properties the stylesheet does set on the card's own box (background, border color, radius, text color, font-size, margin, and padding), plus the header's font weight and the depth badge's background, use the --b4-activity-* tokens. The app sets those in app/theme.css.

React and @copilotkit/react-core are optional peer dependencies of @b4run/ag-ui, and only the ./react subpath needs them. A server that uses the root or ./sse entry installs nothing extra.

To present the same activities your own way, the subpath also exports the pieces behind that array:

• b4PlanActivityRenderer, PlanActivityCard and ActivityChecklist: the plan renderer, and the plain React components behind it (they take a content prop and need no CopilotKit context).
• planActivityContentSchema, to validate plan content before you render it yourself.
• reduceSubagentRuns, the pure reducer behind useSubagentRuns, to build the subagent tree from events in a non-React client.

Plan snapshots replace b4:plan:${runId} (the root agent's) or b4:plan:${call_id} (a subagent's, tagged subagentRunId) and carry the complete todo list. The checklist views show at most eight todos, but the snapshots keep the complete valid lists.

The plan activity is the whole presentation of writeTodos: when a writeTodos call produced its activity, B4.run's AG-UI adapter emits no tool call/result events for that call, so the wildcard tool card never receives it. A task call is an ordinary tool call — the wildcard card renders it, and the subagent it starts shows up in the panel.

The wildcard card is still the fail-open fallback for writeTodos. If the plan activity cannot be produced (no tool-call id, a malformed payload), the ordinary tool events survive and the generic card renders them.

Choose Plan a flight on the empty transcript to see the root plan and the weather and performance progress update before the brief. This live flow uses the B4.run server's model key. The automated browser check only proves V2 transport selection.

Approve memory candidates#Copy link to section: Approve memory candidates

When the coordinator calls remember(), B4.run stores a durable-memory candidate. The runtime exposes candidates over HTTP:

GET  /memory/candidates
POST /memory/candidates/:id/approve
POST /memory/candidates/:id/reject

The browser cannot call those routes directly unless the server sets server.cors. Without it, the B4.run dev server sends no CORS headers, so the example reaches them through a same-origin catch-all, app/api/b4/[...path]/route.ts.

That proxy is allowlisted. app/lib/proxy-allowlist.ts is a pure function listing the exact method and path shape of every route the browser may reach: the three memory routes above plus GET /threads/:id/state and GET /threads/:id/pending_interrupts. Anything else is rejected with 403 and never forwarded. That keeps POST /threads/:id/resume and the rest of the agent surface out of reach in a template every B4.run developer copies.

app/components/MemoryPanel.tsx renders the candidates in the thread rail with Approve and Delete on each. Delete maps to /reject, which is a hard delete on the server. The panel loads on mount and again at the end of every run. remember() lands mid-run, so a memory proposed in the answer you are reading should be reviewable without a reload:

useEffect(() => {
  const controller = new AbortController()
  void load(controller.signal)
  return () => {
    controller.abort()
  }
}, [load])
 
useEffect(() => {
  const subscription = agent.subscribe({
    onRunFinishedEvent: () => {
      void load()
    },
  })
  return () => {
    subscription.unsubscribe()
  }
}, [agent, load])

The panel only reviews candidates. It lists at most three and counts the rest. Browsing, searching, and editing stored memories is still the job of the b4 memory CLI.

Related#Copy link to section: Related