> ## Documentation Index
> Fetch the complete documentation index at: https://docs.fish.audio/llms.txt
> Use this file to discover all available pages before exploring further.

# Wire Protocol

> The realtime message contract between clients and agent sessions, for platforms the SDKs don't cover

Everything the SDKs do rides a small, versioned wire protocol: two JSON message channels, plus the transport's standard transcription and state mechanisms. The transport is WebRTC, built on LiveKit: audio travels as WebRTC [media tracks](https://docs.livekit.io/home/client/tracks/), the message channels ride LiveKit's reliable [data channels](https://docs.livekit.io/home/client/data/messages/), and any platform with a [LiveKit client SDK](https://docs.livekit.io/home/client/connect/) can implement it. This page documents that contract for consumers that cannot use the SDKs: custom native stacks or ports to new platforms.

<Note>
  This is an escape hatch. For web and React apps, use the [Web
  SDK](/agents/deploy/web-sdk) or [React SDK](/agents/deploy/react-sdk)
  instead. They implement everything below (connection, reconnection, message
  parsing, tool dispatch) and stay current as the protocol evolves.
</Note>

## The protocol package

Every message shape on this page is published as TypeScript definitions in `@fishaudio/agent-protocol`, with zero runtime dependencies. This page documents the protocol as of package version **0.1.0**. The package is the source of truth for shapes; this page fixes the semantics.

```bash npm theme={null}
npm install @fishaudio/agent-protocol
```

## Connect to a session

Create a session server-side ([authenticated sessions](/agents/deploy/authenticated-sessions)), then connect to the transport named in the response. [Public agents](/agents/deploy/public-agents) can create sessions without the `Authorization` header.

```bash Create a session theme={null}
curl --request POST https://api.fish.audio/v1/agent/sessions \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{ "agent_id": "YOUR_AGENT_ID" }'
```

```json Response (201) theme={null}
{
  "session_id": "...",
  "expires_at": "2026-07-23T12:34:56Z",
  "max_duration_seconds": 1800,
  "transport": "livekit",
  "livekit_url": "wss://...",
  "token": "<participant JWT>"
}
```

The response is a discriminated union on `transport`. The `livekit` arm carries `livekit_url` and `token`: [connect](https://docs.livekit.io/home/client/connect/) a LiveKit-compatible WebRTC client with them before `expires_at` (the join deadline), then [publish your microphone track and subscribe](https://docs.livekit.io/home/client/tracks/) to the agent's audio track.

<Warning>
  If you receive a `transport` value you don't recognize, fail with an explicit
  "unsupported transport" error. New transport arms may be introduced, each
  with its own payload.
</Warning>

## Channels

Once connected, a session uses these channels. The two `*-event` topics carry reliable [data packets](https://docs.livekit.io/home/client/data/messages/): one complete JSON object per packet.

| Channel                                | Direction      | Carries                                                   |
| -------------------------------------- | -------------- | --------------------------------------------------------- |
| Audio tracks                           | both           | Your microphone up, the agent's speech down               |
| `agent-event` topic                    | agent → client | Control events: client tool calls, tool lifecycle, errors |
| `client-event` topic                   | client → agent | Text turns, activity, interrupts, hangup, tool results    |
| `lk.transcription` text streams        | agent → client | Streaming transcripts for both sides                      |
| `lk.agent.state` participant attribute | agent → client | Agent pipeline state (sticky)                             |

Data-channel message fields are camelCase; REST bodies are snake\_case.

Each channel maps to a standard LiveKit client mechanism:

* **`client-event` (sending)**: publish each message as a UTF-8 JSON payload with [`publishData`](https://docs.livekit.io/home/client/data/messages/), in **reliable** mode, with `topic` set to `client-event`.
* **`agent-event` (receiving)**: listen for the [`DataReceived`](https://docs.livekit.io/home/client/data/messages/) room event and keep only packets whose topic is `agent-event`; decode each payload as one JSON object.
* **Transcription**: register a handler for the `lk.transcription` topic with [`registerTextStreamHandler`](https://docs.livekit.io/home/client/data/text-streams/). Segment semantics are described [below](#transcription-and-agent-state).
* **Agent state**: read `lk.agent.state` from the agent participant's [attributes](https://docs.livekit.io/home/client/state/participant-attributes/) and subscribe to the `AttributesChanged` event for updates.

The complete wiring in the JS SDK (`livekit-client`) looks like this; every LiveKit SDK has equivalent APIs:

```ts Connect and wire up all channels theme={null}
import { Room, RoomEvent } from 'livekit-client';

const room = new Room();

// Register all handlers BEFORE connecting: the agent may already be
// speaking when you join, and `lk.agent.state` is delivered on join.

// Agent events: reliable data packets on the `agent-event` topic
room.on(RoomEvent.DataReceived, (payload, participant, kind, topic) => {
  if (topic !== 'agent-event') return;
  const event = JSON.parse(new TextDecoder().decode(payload));
  // event.type: 'client_tool.call' | 'tool.started' | 'tool.completed' | ...
});

// Transcription: text streams on the `lk.transcription` topic
room.registerTextStreamHandler('lk.transcription', async (reader, participant) => {
  const segmentId = reader.info.attributes?.['lk.segment_id'];
  const isFinal = reader.info.attributes?.['lk.transcription_final'] === 'true';
  const text = await reader.readAll();
  // participant.identity tells you whether this is the user's or the agent's segment
});

// Agent state: sticky `lk.agent.state` participant attribute
room.on(RoomEvent.ParticipantAttributesChanged, (changed, participant) => {
  if ('lk.agent.state' in changed) {
    console.log(participant.attributes['lk.agent.state']);
  }
});

// Agent speech: attach the agent's audio track when it arrives
room.on(RoomEvent.TrackSubscribed, (track) => {
  document.body.appendChild(track.attach());
});

// Now connect, using `livekit_url` and `token` from the
// session-creation response above, and publish your microphone
await room.connect(livekitUrl, token);
await room.localParticipant.setMicrophoneEnabled(true);

// Client events: publish JSON on the `client-event` topic
function send(event: object) {
  return room.localParticipant.publishData(
    new TextEncoder().encode(JSON.stringify(event)),
    { reliable: true, topic: 'client-event' },
  );
}

await send({ type: 'user.message', text: "What's my balance?" });
```

## Agent events: `agent-event`

### `client_tool.call`

The agent wants to run a [client tool](/agents/build/client-tools) in your app.

```json client_tool.call theme={null}
{
  "type": "client_tool.call",
  "callId": "call_abc",
  "toolName": "open_dashboard",
  "params": { "tab": "billing" },
  "expectsResponse": true
}
```

Run the tool, then publish a `client_tool.result` with the same `callId`. While `expectsResponse` is `true`, the agent suspends the model's tool call until your result arrives or the tool's configured timeout elapses (default 30 seconds, configurable from 1 to 120). When `expectsResponse` is `false`, the call is fire-and-forget: the agent continues immediately and any result you send is ignored.

### Tool lifecycle: `tool.started`, `tool.completed`, `tool.failed`

Each tool the agent invokes emits one `tool.started`, resolved by exactly one terminal message (`tool.completed` or `tool.failed`) with the same `callId`. Terminal messages repeat `toolName` and `toolSource`, so a client that missed the start can still render a complete entry.

| Field                                | Type    | Notes                                                                                                        |
| ------------------------------------ | ------- | ------------------------------------------------------------------------------------------------------------ |
| `callId`                             | string  | Correlates a start with its terminal message                                                                 |
| `toolName`                           | string  | The tool name as the model sees it                                                                           |
| `toolSource`                         | string  | Where the tool runs: `client`, `webhook`, `mcp`, `builtin`, `background`, or `unknown`; treat as an open set |
| `input` / `output`                   | string  | JSON-serialized payload, truncated at the source at 4 KB                                                     |
| `inputTruncated` / `outputTruncated` | boolean | Present when the payload was truncated                                                                       |
| `error`                              | string  | `tool.failed` only                                                                                           |

These events are on by default and their payloads travel to the end user's client. Pass `tool_events: false` when creating the session to keep tool data off the wire: the session then receives none of the three.

### `error`

```json error theme={null}
{ "type": "error", "code": "provider_error" }
```

`code` is a coarse category only: `provider_error` (an upstream model or voice provider failed) or `internal_error` (the runtime failed). The message deliberately carries no raw error detail.

## Client events: `client-event`

Publish these on the `client-event` topic. The agent ignores malformed JSON and unknown types.

| Type                 | Effect                                                                |
| -------------------- | --------------------------------------------------------------------- |
| `user.message`       | Injects a text user turn; the agent replies as if the user had spoken |
| `user.activity`      | Signals user activity (typing); suppresses idle re-engagement         |
| `user.interrupt`     | Explicitly interrupts the agent's current speech                      |
| `user.hangup`        | Ends the session gracefully                                           |
| `client_tool.result` | Settles a pending `client_tool.call`                                  |

```json user.message theme={null}
{ "type": "user.message", "text": "What's my balance?" }
```

```json client_tool.result theme={null}
{
  "type": "client_tool.result",
  "callId": "call_abc",
  "result": { "opened": true }
}
```

* `user.message` gets **no server echo**: you already hold the text, so render the bubble locally. Add `"audio": false` to have the agent answer that turn in text only: no speech is synthesized and the reply arrives over transcription. When the field is absent the agent speaks as usual; note the web SDK's `sendUserMessage` sends `"audio": false` unless called with `audio: true`.
* `client_tool.result` may carry `result` (any JSON value) or `"isError": true` to report the tool as failed to the model. Results for unknown or already-settled `callId`s are ignored.

## Transcription and agent state

These ride the transport's built-in mechanisms rather than custom messages.

**Transcription** arrives as [text streams](https://docs.livekit.io/home/client/data/text-streams/) on the `lk.transcription` topic. Segments are identified by the `lk.segment_id` stream attribute; `lk.transcription_final: "true"` marks a segment as final. Roles are distinguished by sender identity: the user's segments are sent under the user's own participant identity, the agent's under the agent participant.

* Agent segments stream incrementally, paced to audio playback. An interrupted segment closes containing only the words actually spoken; there is no residual text.
* User segments are interim until final; each interim update **replaces the entire segment text** under the same segment id.

**Agent state** is published as the sticky `lk.agent.state` [participant attribute](https://docs.livekit.io/home/client/state/participant-attributes/) with values `initializing`, `idle`, `listening`, `thinking`, and `speaking`. Sticky means a client that connects late or reconnects reads the current value immediately. The SDKs derive their three public modes from this attribute plus transcript segment open/close.

## Compatibility rules

1. **Ignore unknown `type` values and unknown fields.** This is required consumer behavior and the foundation of forward compatibility.
2. **Evolution is additive-only.** Published fields never change name or meaning and are never removed; new fields are always optional. A semantic change ships as a new `type`.
3. **No replay.** Data-channel delivery is reliable and ordered within a connection, but after a reconnect or late join, missed messages are gone. Never wait for history. Tool terminal messages repeat their identifying fields, and the state attribute is sticky, precisely to soften this.

## Going further

<CardGroup cols={2}>
  <Card title="Web SDK" icon="js" href="/agents/deploy/web-sdk">
    The supported implementation of this protocol for browsers.
  </Card>

  <Card title="Client tools" icon="wrench" href="/agents/build/client-tools">
    Declare the tools your `client_tool.call` handlers implement.
  </Card>

  <Card title="Authenticated sessions" icon="server" href="/agents/deploy/authenticated-sessions">
    Session creation, API keys, and token handling.
  </Card>

  <Card title="Public agents" icon="globe" href="/agents/deploy/public-agents">
    Let clients create sessions without a backend.
  </Card>
</CardGroup>