Handle Local Media for JavaScript

To send media in a video conference, you must produce local audio and video. Audio or video produced by the current user is called local media. This article describes how to capture local media and send it to remote participants.

You don't need to implement local media because LiveSwitch provides local media that works with major browsers. To use local media, create an instance of fm.liveswitch.LocalMedia. Provide two parameters audio and video, which indicate whether to capture audio and whether to capture video. If you want to capture both, pass true for both parameters.

Version 1.25.0 Improvements

LiveSwitch 1.25.0 introduced significant enhancements to local media handling for improved lip synchronization:

Unified synchronization context - Local audio and video are now generated using proper RTP timestamp management to ensure they remain synchronized, even when tracks are muted and subsequently unmuted
Privacy-conscious video muting - Camera indicator lights turn off when video is muted, assuring users their camera is truly disabled. No video packets are sent when muted (not even black frame deltas), freeing bandwidth for adaptive bitrate algorithms
Smart microphone handling - The microphone remains active when muted, sending empty audio packets to maintain the RTP stream. This enables "muted alert" functionality where applications can notify users when they attempt to speak while muted
Improved device switching - Lip synchronization is maintained when switching video input devices (e.g., flipping from front to back camera)
Enhanced browser interoperability - Proper SDP signaling with msid-semantic attributes ensures browsers can correctly synchronize audio and video tracks, whether sending from a browser, receiving on a browser, or both

Important

In version 1.25.0, multiple local media objects for the same participant are no longer supported. This change is necessary to ensure proper synchronization. Each participant should have only one local media instance.

Creating Local Media

Example:

var audio = true;
var video = true;
var localMedia = new fm.liveswitch.LocalMedia(audio, video);

// For more control, the video parameter can be an instance of fm.liveswitch.VideoConfig. 
// Doing this lets you specify a width, height, and frame rate.
var audio = true;
var video = new fm.liveswitch.VideoConfig(width, height, frameRate);
var localMedia = new fm.liveswitch.LocalMedia(audio, video);

// For more advanced or experimental use cases, the audio and video parameters can be 
// HTML5-compliant MediaTrackConstraints definitions:
var audio = {
    sampleSize: 16,
    channelCount: 2,
    echoCancellation: false
};
var video = {
    width: 320,
    height: 240,
    frameRate: 30
};
var localMedia = new fm.liveswitch.LocalMedia(audio, video);

Note

LiveSwitch passes constraints through to the browser's getUserMedia API. Browser support for some constraints might vary depending on the browser's version.

Muting and Unmuting Local Media

Local audio and video can be muted by updating connection config object:

var config = this.sfuUpstreamConnection.getConfig();
config.setLocalVideoMuted(true);
this.sfuUpstreamConnection.update(config);

Note

We recommend muting and unmuting rather than disabling and re-enabling local media. This allows for faster response, better lipsync and bandwidth adaptation support.

Advanced Media Streams (Not available in 1.25.0)

Note

The ability to pass HTML5 MediaStreams, individual audio tracks, or video tracks directly to fm.liveswitch.LocalMedia is not available in version 1.25.0 and will be re-introduced in version 1.26.0. This feature is useful for advanced use cases such as external blur implementations and custom media processing.

In version 1.26.0, the audio and video parameters will accept an HTML5 media stream. The following code example shows how this will work for streaming media to an HTML canvas:

// This feature  is not available in version 1.25.0
var canvas = document.getElementById('localCanvasSource');
var canvasFrameRate = 3;
if (!canvas) {
    // Create the canvas if it doesn't exist yet.
    canvas = document.createElement('canvas');
    canvas.id = 'localCanvasSource';
    canvas.style.position = 'absolute';
    document.body.appendChild(canvas);
    
    // Load a static image.
    var image = new Image();
    image.onload = function() {
        // Resize the canvas to match the image size.
        canvas.width = image.width;
        canvas.height = image.height;
        canvas.style.left = '-' + image.width + 'px';
        canvas.style.top = '-' + image.height + 'px';
        
        // Draw the initial image.
        var context = canvas.getContext('2d');
        context.drawImage(image, 0, 0, image.width, image.height, 0, 0, canvas.width, canvas.height);
        
        // Refresh the image on a regular interval.
        window.setInterval(function() {
            context.clearRect(0, 0, canvas.width, canvas.height);
            context.drawImage(image, 0, 0, image.width, image.height, 0, 0, canvas.width, canvas.height);
        }, 1000.0 / canvasFrameRate);
    };
    image.src = 'https://v1.liveswitch.fm/images/static.jpg';
}

var audio = false;
var video = canvas.captureStream(canvasFrameRate);
var localMedia = new fm.liveswitch.LocalMedia(audio, video);  // Available in 1.26.0

Important

Until version 1.26.0 is released, please use the boolean flags, VideoConfig, or MediaTrackConstraints approaches shown in the "Creating Local Media" section above. This feature is also available in 1.24.x and earlier.