WebRTC - MediaStream APIs

The MediaStream API (often referred to as getUserMedia) is a cornerstone of WebRTC, designed to provide web applications with easy access to media streams from local input devices like cameras and microphones.

Key aspects of the MediaStream API:

Stream Representation: A real-time flow of media data (audio and/or video) is represented by a MediaStream object.
User Permission: Access to local devices is security-critical. The API enforces a permission model, prompting the user before a web application can start capturing a stream.
Device Selection: The API, in conjunction with navigator.mediaDevices.enumerateDevices(), allows for the selection of specific input devices if multiple cameras or microphones are available.

Each MediaStream object consists of one or more MediaStreamTrack objects. A track represents a single source of media, such as video from a specific camera or audio from a particular microphone.

A MediaStreamTrack itself can have multiple channels (e.g., left and right audio channels for stereo sound). These channels are the most granular components within the MediaStream API model.

MediaStream objects can be utilized in two primary ways:

Rendered locally in HTML <video> or <audio> elements.
Sent to a remote peer via an RTCPeerConnection object.

Using the MediaStream API: A Basic Example

Let’s create a simple WebRTC application. It will display a <video> element, request user permission to access the camera and microphone, and then show the live video stream in the browser. Create an index.html file:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>MediaStream API Demo</title>
    <style>
        video { width: 320px; height: 240px; border: 1px solid black; background-color: #333; }
        body { font-family: sans-serif; padding: 20px; }
        .controls button { margin: 5px; padding: 8px 12px; }
    </style>
</head>
<body>
    <h1>My Live Video Stream</h1>
    <video id="localVideo" autoplay playsinline muted></video>
    <!-- 'autoplay' starts playback, 'playsinline' for mobile, 'muted' prevents local echo -->

    <div class="controls">
        <button id="btnGetAudioTracks">Log Audio Tracks</button>
        <button id="btnGetVideoTracks">Log Video Tracks</button>
        <button id="btnGetTrackById">Log First Video Track by ID</button>
        <button id="btnGetTracks">Log All Tracks</button>
        <button id="btnRemoveAudioTrack">Remove First Audio Track</button>
        <button id="btnRemoveVideoTrack">Remove First Video Track</button>
        <button id="btnStopAllTracks">Stop All Tracks (End Stream)</button>
    </div>

    <script src="client.js"></script>
</body>
</html>

Then, create client.js:

const localVideo = document.querySelector('#localVideo');
let localStream; // To store the MediaStream object

async function startMedia() {
    if (!navigator.mediaDevices || !navigator.mediaDevices.getUserMedia) {
        alert('getUserMedia() is not supported by your browser.');
        console.error('getUserMedia() is not supported.');
        return;
    }

    try {
        // Request video and audio access
        const constraints = { video: true, audio: true };
        localStream = await navigator.mediaDevices.getUserMedia(constraints);
        console.log('Got MediaStream:', localStream);

        // Attach the stream to the video element
        localVideo.srcObject = localStream;

        // Example: Log when the stream becomes inactive
        localStream.oninactive = () => {
            console.log('Stream became inactive');
        };

    } catch (error) {
        console.error('Error accessing media devices.', error);
        alert(`Error accessing media devices: ${error.name}: ${error.message}`);
        if (error.name === 'NotFoundError' || error.name === 'DevicesNotFoundError') {
            alert('No camera/microphone found.');
        } else if (error.name === 'NotAllowedError' || error.name === 'PermissionDeniedError') {
            alert('Permission to access camera/microphone was denied.');
        } else if (error.name === 'OverconstrainedError' || error.name === 'ConstraintNotSatisfiedError') {
            alert('The specified constraints could not be satisfied by available devices.');
        } else {
            alert('An unknown error occurred while accessing media devices.');
        }
    }
}

// --- Button Event Listeners for API testing ---
document.getElementById('btnGetAudioTracks').addEventListener('click', () => {
    if (!localStream) { alert('Stream not started'); return; }
    const audioTracks = localStream.getAudioTracks();
    console.log('Audio Tracks:', audioTracks);
    if (audioTracks.length > 0) console.log('First audio track settings:', audioTracks[0].getSettings());
});

document.getElementById('btnGetVideoTracks').addEventListener('click', () => {
    if (!localStream) { alert('Stream not started'); return; }
    const videoTracks = localStream.getVideoTracks();
    console.log('Video Tracks:', videoTracks);
    if (videoTracks.length > 0) console.log('First video track settings:', videoTracks[0].getSettings());
});

document.getElementById('btnGetTrackById').addEventListener('click', () => {
    if (!localStream || localStream.getVideoTracks().length === 0) {
        alert('Stream not started or no video tracks'); return;
    }
    const firstVideoTrackId = localStream.getVideoTracks()[0].id;
    const trackById = localStream.getTrackById(firstVideoTrackId);
    console.log(`Track by ID (${firstVideoTrackId}):`, trackById);
});

document.getElementById('btnGetTracks').addEventListener('click', () => {
    if (!localStream) { alert('Stream not started'); return; }
    const allTracks = localStream.getTracks();
    console.log('All Tracks:', allTracks);
});

document.getElementById('btnRemoveAudioTrack').addEventListener('click', () => {
    if (!localStream || localStream.getAudioTracks().length === 0) {
        alert('Stream not started or no audio tracks to remove'); return;
    }
    const audioTrack = localStream.getAudioTracks()[0];
    localStream.removeTrack(audioTrack);
    console.log('Removed audio track:', audioTrack);
    console.log('Remaining audio tracks:', localStream.getAudioTracks());
    // Note: Removing the track from the stream doesn't automatically stop the track or release the device.
    // To fully stop it, you might need to call track.stop() if it's the last use of that track.
});

document.getElementById('btnRemoveVideoTrack').addEventListener('click', () => {
    if (!localStream || localStream.getVideoTracks().length === 0) {
        alert('Stream not started or no video tracks to remove'); return;
    }
    const videoTrack = localStream.getVideoTracks()[0];
    localStream.removeTrack(videoTrack);
    console.log('Removed video track:', videoTrack);
    console.log('Remaining video tracks:', localStream.getVideoTracks());
     // The video element might go black or show a paused frame.
});

document.getElementById('btnStopAllTracks').addEventListener('click', () => {
    if (!localStream) { alert('Stream not started'); return; }
    localStream.getTracks().forEach(track => {
        track.stop();
        console.log(`Stopped track: ${track.kind} - ${track.label}`);
    });
    console.log('All tracks stopped. Stream is now inactive.');
    localVideo.srcObject = null; // Clear the video element
    // localStream is now inactive. You might want to nullify it: localStream = null;
});

// Start the media capture when the script loads
startMedia();

In this updated client.js:

We use navigator.mediaDevices.getUserMedia(), which is Promise-based and the current standard.
The obtained MediaStream (localStream) is assigned to localVideo.srcObject to display it. This replaces the older URL.createObjectURL() method for streams.
Error handling for getUserMedia is more robust, catching specific error types.
The <video> element has playsinline (important for iOS Safari to prevent fullscreen) and muted (crucial for local video to prevent audio feedback/echo if audio is also captured).
Event listeners are added for buttons to demonstrate various MediaStream API methods.

To run this, open index.html in a modern browser (preferably from a local web server, e.g., using npx serve or Python’s http.server). The browser will ask for permission to use your camera and microphone. Once granted, your video should appear. The original tutorial showed an image of this permission prompt and the resulting video stream.

MediaStream API Reference

Properties

active (read-only boolean): Returns true if the MediaStream is active (i.e., at least one of its tracks is not ended), or false otherwise.
id (read-only string): A unique identifier (GUID) for the MediaStream object.
ended (read-only boolean, Deprecated): Returns true if the ended event has been fired, meaning the stream has completely finished. Use active instead.

The original tutorial included an image showing these properties in a browser’s console. You can inspect the localStream object in your developer console to see these values.

Event Handlers (on a `MediaStream` object)

onactive: Fired when a MediaStream object becomes active. (Less commonly used directly; usually check active property).
oninactive: Fired when a MediaStream object becomes inactive (e.g., all its tracks have ended).
onaddtrack: Fired when a new MediaStreamTrack object is added to this MediaStream (e.g., by addTrack()).
onremovetrack: Fired when a MediaStreamTrack object is removed from this MediaStream (e.g., by removeTrack()).
onended (Deprecated): Fired when streaming terminates. Listen for ended events on individual tracks or check MediaStream.active.

Methods

addTrack(track): Adds the given MediaStreamTrack to this MediaStream.
clone(): Returns a new MediaStream object which is a clone of this stream. The new stream will have a new unique id, but will contain the same set of MediaStreamTrack objects (by reference).
getAudioTracks(): Returns an array of all MediaStreamTrack objects in this stream that have their kind property as "audio".
getVideoTracks(): Returns an array of all MediaStreamTrack objects in this stream that have their kind property as "video".
getTracks(): Returns an array of all MediaStreamTrack objects in this stream, regardless of kind.
getTrackById(trackId): Returns the MediaStreamTrack object from this stream that has the specified id. Returns null if no track matches.
removeTrack(track): Removes the specified MediaStreamTrack from this MediaStream. This does not stop the track itself; it just removes it from this particular stream.

The button-driven examples in client.js above demonstrate these methods. For instance, clicking ‘Log Audio Tracks’ uses getAudioTracks(). Clicking ‘Remove First Video Track’ uses getVideoTracks() to get the track and then removeTrack() to remove it. The original tutorial included images of console output after clicking similar buttons, showing arrays of tracks or specific track information.

A MediaStreamTrack object also has important methods like stop() (which stops the track and releases the hardware resource) and properties like kind ('audio' or 'video'), label (descriptive name), and enabled (to mute/unmute or hide/show).

Summary

This chapter provided an overview of the MediaStream API and demonstrated its basic usage. You should now have a clearer understanding of how to access local media devices and manage media streams and tracks, which are fundamental building blocks for any WebRTC application involving audio or video.

WebRTC - MediaStream APIs

WebRTC - MediaStream APIs

Using the MediaStream API: A Basic Example

MediaStream API Reference

Properties

Event Handlers (on a MediaStream object)

Methods

Summary

Event Handlers (on a `MediaStream` object)