Use speech recognition on an initialized Voice session. Configure input and output formats separately: asr.start.audio selects microphone input; client.start.config.audioEncoding selects agent output.
Start and stop recognition
Register listeners before starting. Wait for asr.started before forwarding audio. Stop forwarding when asr.stopped arrives.
Send raw binary frames with asr.audio.send. Base64 and JSON strings are rejected. A browser MediaRecorder container is not raw PCM; encode audio in the acknowledged format before sending it.
Choose turn handling
Partial transcripts can change; the committed asr.transcript replaces them. Speech-recognition provider settings come from the selected Environment. With stopWhenIdle: false, recognition remains open through silence. For output and interruption handling, see Playback and responses.
Recognition events
asr.start
Client → server. Open speech recognition on the ready session. The server replies with asr.started or asr.stopped. Speech-recognition provider settings come from the selected Environment.
Supported input formats:
SocketASRTurnsEnum: "client", "server".
asr.audio.send
Client → server. Send a binary frame in the format acknowledged by asr.started. ArrayBuffer, typed-array and Buffer frames are accepted; JSON strings and base64 text are not.
Payload: a raw binary audio frame in the format selected by asr.start.
asr.started
Server → client. Recognition is ready. The payload echoes the audio format and turn-handling options the server applied. Begin forwarding audio after this event.
Supported input formats:
SocketASRTurnsEnum: "client", "server".
asr.stop
Client → server. Close speech recognition. The server replies with asr.stopped.
Payload: {}.
asr.stopped
Server → client. Recognition has stopped, or could not start. Stop forwarding audio and inspect the reason: client, idle, max_duration, error or quota.
SocketASRStopReasonEnum: "client", "idle", "max_duration", "error", "quota".
asr.transcript
Server → client. A committed transcription of the customer’s utterance. Server turns submit it automatically; client turns require your application to submit it.
asr.transcript.partial
Server → client. Revisable words heard so far. Use them for a live transcript display; the committed asr.transcript supersedes them.
words item fields: