Simulcast

Overview

Simulcast is short for "simultaneous broadcast." It's used in WebRTC to improve the overall quality of a group conference.

Instead of sending just one video stream from the source to the server and then to each participant, the server also creates copies ("encodings") of the same video but encoded to different quality targets. Typically, a high-quality (full bitrate), medium-quality and low-quality (reduced bitrate) streams are encoded.

Consider a group call with Alice, Bob, Carol, and Dan:

Alice is sending a video using a bitrate of 2 Mbps.
Bob and Dan have reliable internet connections and can receive Alice's video at 2 Mbps.
Carol, however, has a limited internet connection and can only sustain inbound video at 1 Mbps.

In this scenario, to keep Carol from dropping the call, you must coordinate with Alice to reduce her outbound bitrate to 1 Mbps. Unfortunately for Bob and Dan, they must also see Alice at the reduced bitrate. This negative side effect is amplified and becomes more and more noticeable as you add more participants. With simulcast, you can send Carol a low-quality encoding while allowing Bob and Dan to continue receiving the high-quality encoding.

Warning

Please be aware that Client-Side Simulcast has limitations. If the video stream sender has poor network connectivity, all receivers will experience poor performance. Client-Side Simulcast depends on the sender having a strong network to handle multiple video encodings being sent. In such cases, Server-Side Simulcast is preferred to maintain high-quality video streams for receivers.

Warning

Simulcast is not supported on P2P connections. In two-way calls, the bitrate adapts to each receiver individually, with no side effects since there are no additional receivers. Only SFU connections can benefit from simulcast.

How does Simulcast Improve Stream Quality?

Simulcast is a powerful tool for providing optimal video stream quality. It works by simultaneously sending multiple feeds of different quality from server to the receiving clients, dynamically enabling and disabling them depending to the network conditions experienced by the individual receivers.

Simulcast, by itself, can't resolve all quality issues for all network conditions. However, you can use it to fix most of the quality issues you encounter. There are other mechanisms that also affect overall stream quality. An alternative to simulcast that improves overall stream quality is Adaptive Video Resolution, where you adjust the video stream's resolution, constraining it to the actual solution of the receivers' view. For example, if you send HD video at a resolution of 1280 x 720, but all the receivers display the downstream in a smaller view because the app layout dictates demoting participant views during a screen share, you can adjust the sender's resolution to not waste bandwidth.

LiveSwitch continually improves bandwidth management and adaptation policies and capabilities. Contact Support if you have a use case that you want to improve.

Configure Simulcast

To enable simulcast in the server, you have three options: through the server console, using the REST API, or by setting environment variables.

Simulcast can be enabled at the Deployment, Application, or Channel level. Lower-level configurations override higher-level ones. For example, if simulcast is disabled for a deployment, it can still be enabled for a specific channel or channel pattern.

You can configure the following values:

"Enabled": Indicates whether simulcast is enabled. Use the enabledV2 variable for the REST API.
"Video Encoding Count": Indicates the number of video encodings enabled for simulcast. This includes the primary encoding. A value of 0 disables simulcast, while 3 is recommended.
"Degradation Preference": Determines how quality is degraded for simulcast substreams. Options include "Automatic" (recommended), "Balanced", "Frame Rate", and "Resolution".
"Max Simulcast Upstreams Per Channel": Sets the maximum number of publishers allowed on a channel when simulcast is enabled. The recommended value is 3.
"Allowed": Indicates whether simulcast is allowed on a channel, deployment, or application. This controls whether simulcast can be enabled at a particular level.
"Manual Simulcast Encodings": By default, the server determines simulcast encodings based on degradation preference and encoding count. However, you can specify encodings as fractions of resolution and frame rate relative to the primary encoding. These settings override degradation preferences and encoding count.
- Example: If the primary encoding is 1080p at 30 fps, specifying {(FrameRateScale = 0.5, ResolutionScale = 0.5), (FrameRateScale = 0.25, ResolutionScale = 0.5)} will result in three encodings: the original/primary encoding, a simulcast substream at 960px540p and 15 fps, and another simulcast substream at 960px540p and 8 fps.

Video Retention

In certain scenarios, Simulcast may not suffice for users experiencing poor network conditions. For instance, a user might be on the lowest encoding layer yet still struggle with video reception due to network constraints.

To maintain optimal user experience even during challenging network conditions, Simulcast can leverage our Video Retention Policy. This feature intelligently notifies users when their connection quality necessitates switching to the lowest video encoding, and recommends disabling video to preserve communication quality.

As network issues may be transient, we also provide options to automatically re-enable video when conditions improve. The following sections detail the various configuration parameters available for tailoring Video Retention to your specific Simulcast implementation.

Video Retention Policy

You can manage this issue using the VideoRetentionPolicy on the Downstream Connection. The available options are listed below, with the default setting being DynamicRetention.

Value	Description
AlwaysRetain	Video will always be retained regardless of connection quality. The video will not be turned off under any circumstances.
DynamicRetention	Video will be dynamically enabled or disabled based on the `VideoEnabledUpdateConnectionPolicy` function. While this function has a default configuration, it can be overridden with your custom function. *Further details are provided below.
NeverRetain	Video will be disabled upon the first indication of failure and will remain off. When the connection detects the user's initial struggle, the video will be turned off and will not be re-enabled.

Here are some examples on how to set this policy.

downstreamConnection.VideoRetentionPolicy = VideoRetentionPolicy.DynamicRetention;

downstreamConnection.setVideoRetentionPolicy(VideoRetentionPolicy.DynamicRetention);

downstreamConnection?.setVideoRetentionPolicy(FMLiveSwitchVideoRetentionPolicy.dynamicRetention)

downstreamConnection.setVideoRetentionPolicy(fm.liveswitch.VideoRetentionPolicy.DynamicRetention);

Video Enable Update Connection Policy

When using the VideoRetentionPolicy.DynamicRetention policy, the video will be dynamically re-enabled based on a default function or a custom function of your choosing. This function evaluates the number of attempts made to restore the video and returns an integer value indicating, in seconds, the delay before the next attempt. A value of -1 will stop further attempts, keeping the video off.

You can override the default behavior using the VideoEnabledUpdateConnectionPolicy function. The default function will attempt to reconnect the video up to 3 times, with an initial wait time of 30 seconds for the first attempt, increasing by 30 seconds with each subsequent attempt (i.e., 30 seconds + (number of attempts * 30 seconds)).

Below are examples of how to override this function. The custom function will take one integer parameter, representing the number of attempts made to reconnect the video, and will return an integer value indicating the wait time in seconds before the next attempt.

private int MyCustomMethod(int attempts)
{
    if(attempts >= 3)
    {
        return -1;
    }

    return 30 + (attempts * 30);
}

// Further down in your openSfuDownstreamConnection method
downstreamConnection.VideoEnabledUpdateConnectionPolicy = MyCustomMethod;

private int MyCustomFunction(int attempts)
{
    if(attempts >= 3)
    {
        return -1;
    }
    return 30 + (attempts * 30);
}

// Further down in your openSfuDownstreamConnection method
downstreamConnection.setVideoEnabledUpdateConnectionPolicy(this::MyCustomFunction);

downstreamConnection?.setVideoEnabledUpdatePolicy(FMLiveSwitchFunction1(block: { (obj:Any!) in
    let attempts = obj as! Int
    if(attempts >= 3)
    {
        return -1;
    }
    
    return 30 + (attempts * 30)
}))

downstreamConnection.setVideoEnabledUpdateConnectionPolicy(function (attempts) {
    if (attempts >= 3) {
        return -1;
    }

    return 30 + (attempts * 30);
});

Server Notification of Video Enabled/Disabled

To ensure a seamless user experience, your application should be notified whenever a video stream is enabled or disabled. You can attach an event to the Downstream Connection by adding it to the OnVideoEnabledChange action. This action will return two parameters: the downstream connection and a boolean value indicating whether the video is enabled or disabled.

downstreamConnection.OnVideoEnabledChange += (connection, videoEnabled) =>
{
  if(videoEnabled)
  {
    // TODO: Add view back in
  }
  else
  {
    // TODO: Remove view
  }
};

downstreamConnection.addOnVideoEnabledChange((connection,videoEnabled) -> {
    if(videoEnabled)
    {
        // TODO: Add view back in
    }
    else
    {
        // TODO: Remove view
    }
});

downstreamConnection?.addOnVideoEnabledChange({ (conn:FMLiveSwitchManagedConnection?, videoEnabled:Bool) in
    if(videoEnabled)
    {
        //TODO: Add view back in
    }
    else
    {
        //TODO: Remove view
    }
})

downstreamConnection.addOnVideoEnabledChange((connection, videoEnabled) => { 
  if(videoEnabled)
  {
    // TODO: Add view back in
  }
  else
  {
    // TODO: Remove view
  }
});