Guides / Receiving media
Receiving media
The session exposes inbound media as async iterators of PyAV frames. Frames arrive in real time; when a consumer falls behind, the SDK drops the oldest frame rather than growing latency.
Video frames
Each frame is a PyAV VideoFrame. Convert it to a NumPy array in the pixel format
you need:
async with client.connect() as session:
async for frame in session.video_frames():
img = frame.to_ndarray(format="rgb24") # (H, W, 3) uint8
# ... hand `img` to your renderer, encoder, or ML pipelineUse format="bgr24" when feeding OpenCV.
Audio frames
Each frame is a PyAV AudioFrame carrying 48 kHz stereo int16 PCM:
async for frame in session.audio_frames():
samples = frame.to_ndarray() # int16 PCM
sample_rate = frame.sample_rate # 48000Consuming both at once
Run the two iterators concurrently:
import asyncio
async with client.connect() as session:
async def video():
async for frame in session.video_frames():
handle_video(frame.to_ndarray(format="rgb24"))
async def audio():
async for frame in session.audio_frames():
handle_audio(frame.to_ndarray())
await asyncio.gather(video(), audio())The iterators end when the session closes, so async for loops exit
cleanly on teardown. You don't need to poll is_active inside the
loop.
Saving a clip
A quick example that writes the video to disk with OpenCV (install the display
extra):
import cv2, numpy as np
async with client.connect() as session:
await session.talk("Recording a short sample now.")
writer = None
async for frame in session.video_frames():
img = frame.to_ndarray(format="bgr24")
if writer is None:
h, w = img.shape[:2]
writer = cv2.VideoWriter("clip.mp4", cv2.VideoWriter_fourcc(*"mp4v"), 25.0, (w, h))
writer.write(img.astype(np.uint8))
if writer:
writer.release()