Qwen Audio Realtime

qwen-audio-3.0-realtime-plus and qwen-audio-3.1-realtime-plus accept text and audio over WebSocket and stream audio, text, and tool events. Availability depends on account access and platform configuration.

Connection

wss://cubicspaces.cloud/v1/realtime?model=qwen-audio-3.0-realtime-plus
wss://cubicspaces.cloud/v1/realtime?model=qwen-audio-3.1-realtime-plus

Set Authorization: Bearer <CUBICSPACES_API_KEY> during the handshake. Use a WebSocket client; this model does not use the Chat Completions endpoint.

Run the Python examples on your server. A browser's native WebSocket cannot set arbitrary handshake headers. For a browser voice application, establish the authenticated connection from your backend and relay events; keep the platform API key on the backend.

Runnable example

Download qwen_realtime.py and install pip install websocket-client. In PowerShell:

$env:CUBICSPACES_API_KEY = "your platform API key"
python qwen_realtime.py --text "Hello. Please introduce yourself briefly."
python qwen_realtime.py --model qwen-audio-3.1-realtime-plus --text "Hello. Please introduce yourself briefly."
python qwen_realtime.py --mode audio --input question.wav --output answer.wav
python qwen_realtime.py --mode search --text "Search for today's weather in Hangzhou and give the sources."
python qwen_realtime.py --mode function

On macOS / Linux, use export CUBICSPACES_API_KEY='your platform API key'. The four modes cover text-to-speech, audio-file-to-speech, web search, and a real client-side function call. Output is a playable 24 kHz WAV file. The function example calculates 17 + 25, returns 42, and receives the model's spoken answer. Search and function modes open separate connections.

Audio mode requires a 16 kHz, mono, 16-bit PCM WAV file. Convert a recording with:

ffmpeg -i recording.wav -ar 16000 -ac 1 -c:a pcm_s16le question.wav

The script extracts PCM samples and sends 20 ms chunks without the WAV header. For a microphone application, capture and send audio continuously while receiving events and playing output concurrently.

Text input and audio output

Both versions use the same WebSocket event protocol. The default voice is longanqian for 3.0 Plus and longanqian_v3.1 for 3.1 Plus. The script selects the matching default with --model; use --voice to choose a different voice. Set voice in the first session.update; later changes are ignored.

To migrate the example below to 3.1, use model=qwen-audio-3.1-realtime-plus in the URL and voice: longanqian_v3.1 in the session. Audio remains 16 kHz PCM for input and 24 kHz PCM for output.

After session.created, send:

{
  "type": "session.update",
  "session": {
    "modalities": ["text", "audio"],
    "voice": "longanqian",
    "instructions": "Reply briefly.",
    "turn_detection": {"type": "smart_turn"}
  }
}

Wait for session.updated, then send these as separate messages:

{"type":"conversation.item.create","item":{"type":"message","role":"user","content":[{"type":"input_text","text":"Hello"}]}}
{"type":"response.create"}

Decode the delta field of each response.audio.delta event from Base64 and append it to the audio playback buffer. Output is 24 kHz, 16-bit, mono raw PCM, without a WAV header. response.audio_transcript.delta contains the accompanying text. For text-only output, set modalities to ["text"] and consume response.text.delta.

Audio input

Input must be 16 kHz, 16-bit, mono PCM. Encode each chunk as Base64 and send:

{"type":"input_audio_buffer.append","audio":"<Base64 PCM>"}

Input and output use different sample rates. Configure them separately in your client.

With server_vad or smart_turn, the service detects turn boundaries. For manual control, set turn_detection to null in the initial session update. After appending audio, send these two messages:

{"type":"input_audio_buffer.commit"}
{"type":"response.create"}

Set turn detection before sending the first audio chunk.

Events and cancellation

• session.created and session.updated: session setup.
• response.audio.delta: an audio chunk.
• response.text.delta and response.audio_transcript.delta: incremental text.
• response.function_call_arguments.delta and .done: tool arguments. Run the tool locally, submit a function_call_output item through conversation.item.create, then send response.create.
• response.done: a completed, cancelled, or failed turn; inspect response.status.
• error: inspect the error object for details.

Send response.cancel to cancel a running response. Clear buffered playback when the user interrupts. Audio, text, VAD, tool, and extension event fields retain their original format, including explicit false, 0, and null values.

This endpoint supports WebSocket, not WebRTC or AOQ.

Interruptions and continuation

Send response.cancel during generation to receive response.done with status: cancelled and status_details.reason: client_cancelled. The connection remains available: add the next message and send response.create to continue.

In server_vad mode, new speech during generation cancels the current response with reason turn_detected. The service then processes the new speech and generates the next response automatically. Stop playback and clear queued audio immediately when the user interrupts.

A cancelled response may omit usage. Do not treat this omission as a connection failure or assume every response.done includes that field.

For explicit cancellation, send the first message below and wait for the cancelled response's response.done. Then send the other two messages to continue:

{"type":"response.cancel"}
{"type":"conversation.item.create","item":{"type":"message","role":"user","content":[{"type":"input_text","text":"Change the topic. Just say OK."}]}}
{"type":"response.create"}

For automatic speech interruption, configure the initial session with:

{"type":"session.update","session":{"modalities":["text","audio"],"voice":"longanqian","turn_detection":{"type":"server_vad","silence_duration_ms":500}}}

Keep sending microphone audio. On input_audio_buffer.speech_started, stop playback and clear queued output. Use each audio event's response_id to discard late chunks from the cancelled response so they cannot play with the next reply. Automatic mode does not need a commit and response.create for each utterance. Do not trigger another generation while a response is active.

Web search and Function Calling

Enable search with enable_search: true in session.update. Set search_options: {"enable_source": true} to request sources. The model decides whether to search; inspect search_info.search_results in the final response. Search does not guarantee factual accuracy.

Do not enable tools and enable_search together. Define tools as {"type":"function","function":{"name":"...","parameters":{...}}}. The client executes the function, returns a function_call_output with the same call_id, and sends response.create to generate the reply.

Example search configuration; wait for session.updated before submitting the question:

{"type":"session.update","session":{"modalities":["text","audio"],"voice":"longanqian","enable_search":true,"search_options":{"enable_source":true}}}

The downloadable script's --mode function includes tool declarations, complete argument collection, argument validation, execution, result submission, and spoken output. The result's output must be a JSON string. Use the call_id received for that call, not a newly generated ID. Execute only functions explicitly supported and allowed by your client.

Troubleshooting

Symptom	Check
Handshake does not return HTTP 101	Verify the WebSocket URL, platform API key, and model access.
Distorted or accelerated audio	Input is 16 kHz and output is 24 kHz; both use mono, 16-bit little-endian PCM. Do not send a WAV header.
No response after audio input	Manual mode requires commit followed by response.create. Automatic mode needs a complete utterance and trailing silence.
Old audio continues after cancellation	Stop the client player, clear its queue, and discard old chunks by response_id.
No final answer after a tool call	Collect complete arguments, execute the function, return function_call_output with the matching call_id, then send response.create.

Inspect error objects and each response.done status. Reconnecting creates a new session: resend session configuration and do not assume the previous conversation resumes automatically.