RTMP Stream Layout

In an RTMP stream, you can prioritize specific video content over other video content so that viewers have the best view of the most important content. The customers with RTMP authorized can set the layout of the RTMP stream.

Note

All upstream audio connections will be included in the mix. Only a maximum of 5 upstream connections will be visible.

The Channel object has new functions for updating the LayoutZone and LayoutPriority on Connections in the channel.

Channel.UpdateLayout(ChannelLayoutConfig layoutConfig) - Sets the zone and priority of all upstream connections in the channel.
Channel.UpdateUserLayout(string userId, ChannelUserLayoutConfig userLayoutConfig) - Sets the zone and priority of all upstream connections in the channel with the specified userId.
Channel.UpdateDeviceLayout(string userId, string deviceId, ChannelDeviceLayoutConfig deviceLayoutConfig) - Sets the zone and priority of all upstream connections in the channel with the specified userId and deviceId.
Channel.UpdateClientLayout(string userId, string deviceId, string clientId, ChannelClientLayoutConfig clientLayoutConfig) - Sets the zone and priority of all upstream connections in the channel with the specified userId, deviceId, and clientId.
Channel.UpdateClientLayout(ClientInfo remoteClientInfo, ChannelClientLayoutConfig clientLayoutConfig) - Sets the zone and priority of all upstream connections in the channel that match the ClientInfo.
Channel.UpdateConnectionLayout(string userId, string deviceId, string clientId, string connectionId, ChannelConnectionLayoutConfig connectionLayoutConfig) - Sets the zone and priority for the upstream connection in the channel with the specified userId, deviceId, clientId, and connectionId.
Channel.UpdateConnectionLayout(ConnectionInfo remoteConnectionInfo, ChannelConnectionLayoutConfig connectionLayoutConfig) - Sets the zone and priority for the upstream connection in the channel that matches the specified ConnectionInfo.

Types of Stream Layout

RTMP supports the following types of stream layout:

Full
- A full stream takes 100% of the screen.
- Full overrides all other selections.

Promoted
- A promoted stream is displayed in the left 75% of the screen. Four video streams can be displayed on the remaining 25% of the screen on the right-hand side.
- There can be multiple promoted streams. The standard grid layout is applied to multiple promoted streams in order of joining in the 75% of the display reserved for promoted streams.
- If the only available stream is promoted, it defaults to 100% of the screen.
- Size of video box is consistent based on spacing for four boxes. If there are less than four streams, the available streams are centered vertically on the right side.
- Appearance in the boxes is regulated by stream priority assignment. Priorities 1 through 4 show from top to bottom.
- If there are no priority assignment 1-4 streams, streams are shown by MCU default (that's, expected join order).
- If there are multiple priority assignment x streams, they are considered a tie and displayed based on priority order and order of connection until the 5 connection limit is reached.

Grid
- Grid is the default layout and used when there is no promoted or full stream.
- Users show in order of priority, if assigned.
- If there is no priority assigned, users show in connection order (that's, existing order of MCU inputs).

Label a Stream as Full or Promoted

To label a stream as full or promoted, you must perform an update on the Connection to set the LayoutZone on the connection. For RTMP, the zone can be any of the predefined zones in the RTMP layout. If the specified zone isn't part of the layout, then the connection is treated as being in the default zone.

You can assign priority to a stream; priority value can be any integer. Connections within a zone are sorted in an ascending order by the LayoutPriority. For RTMP, this results in the lowest priority always at the top left of any zone. If a zone has multiple rows, then connections are sorted left to right per row.

Note

Each layout update takes both a zone and priority. If you only want to change the zone, you still must send the existing priority; if you are only changing priority, you must also include the current zone.

Create an Upstream Connection with Zone and Priority

Set the LayoutZone or LayoutPriority on new connections as below:

var audioStream = new FM.LiveSwitch.AudioStream(localMedia, null);
var videoStream = new FM.LiveSwitch.VideoStream(localMedia, null);
var connection = channel.CreateSfuUpstreamConnection(audioStream, videoStream);
connection.LayoutZone = ...
connection.LayoutPriority = ...
connection.Open().Then((result) =>
{
    Console.WriteLine("upstream connection established");
}).Fail((Exception ex) =>
{
    Console.WriteLine("an error occurred");
});

fm.liveswitch.AudioStream audioStream = new fm.liveswitch.AudioStream(localMedia, null)
fm.liveswitch.VideoStream videoStream = new fm.liveswitch.VideoStream(localMedia, null)
fm.liveswitch.SfuUpstreamConnection connection = channel.createSfuUpstreamConnection(audioStream, videoStream);
connection.setLayoutZone(...)
connection.setLayoutPriority(...)
connection.open().then((Object result) -> {
    System.out.println("upstream connection established");
}).fail((Exception ex) -> {
    System.out.println("an error occurred");
});

var audioStream = FMLiveSwitchAudioStream(localMedia: localMedia, remoteMedia: nil)
var videoStream = FMLiveSwitchVideoStream(localMedia: localMedia, remoteMedia: nil)
var connection:FMLiveSwitchSfuUpstreamConnection = channel.createSfuUpstreamConnection(audioStream: audioStream, videoStream: videoStream)
connection.setLayoutZone(...)
connection.setLayoutPriority(...)
connection.open().then(resolveBlock: { (result:NSObject) in
    print("upstream connection established")
}).fail(rejectBlock: { (ex:NSException) in
    print("an error occurred")
})

var audioStream = new fm.liveswitch.AudioStream(localMedia, null);
var videoStream = new fm.liveswitch.VideoStream(localMedia, null);
var connection = channel.createSfuUpstreamConnection(audioStream, videoStream);
connection.setLayoutZone(...)
connection.setLayoutPriority(...)
connection.open().then(function(result) {
    console.log("upstream connection established");
}).fail(function(ex) {
    console.log("an error occurred");
});

For updating a specific connection, refer to the examples below. You can modify each example to update connections.

Change Channel.UpdateConnectionLayout to the desired method you want to use for the connections you want to target.
- UpdateLayout - All connections
- UpdateUserLayout - Connections for a user
- UpdateDeviceLayout - Connections for a device
- UpdateClientLayout - Connections for a client
Change ChannelConnectionLayoutConfig to the configuration class for the update you are performing.
- ChannelLayoutConfig - All connections
- ChannelUserLayoutConfig - Connections for a user
- ChannelDeviceLayoutConfig - Connections for a device
- ChannelClientLayoutConfig - Connections for a client

Each configuration class above has a Zone and Priority property to be set. In each configuration class, you have the option to specify excluded connections, devices, users, or clients depending on the level you are targeting.

The pattern here is the same as Channel.Update and Channel.Kick.

Update a Connection Layout

// Prior to registration, ensure you create a claim with the CanUpdateLayout property set to true.
var channelClaims = new[] { new ChannelClaim(ChannelId) { CanLayoutUpdate = true } };

// Create a registration token with your ChannelClaims then call `client.Register(token)`.

// Create a new ChannelConnectionLayoutConfig, and set the Zone and Priority as desired
var layoutConfig = new ChannelConnectionLayoutConfig();
layoutConfig.Zone = FM.LiveSwitch.RtmpLayoutZones.PromotedLeft;
layoutConfig.Priority = 0;

// Send the update - the channel object here is returned when calling `client.Register(token)`.
// The userId, deviceId, clientId, and connectionId must match a connection in the channel.
channel.UpdateConnectionLayout(userId, deviceId, clientId, connectionId, layoutConfig);

// Prior to registration, ensure you create a channel claim with the CanUpdateLayout property set to true.
final ChannelClaim[] channelClaims = new ChannelClaim[]{new ChannelClaim(channelId)};
channelClaims[0].setCanUpdateLayout(true);

// Create a registration token with your ChannelClaims then call `client.register(token)`.

// Create a new ChannelConnectionLayoutConfig, and set the Zone and Priority as desired
final ChannelConnectionLayoutConfig layoutConfig = new ChannelConnectionLayoutConfig();
layoutConfig.setZone(RtmpLayoutZones.getPromotedLeft());
layoutConfig.setPriority(new NullableInteger(0));

// Send the update - the channel object here is returned when calling `client.register(token)`.
// The userId, deviceId, clientId, and connectionId must match a connection in the channel.
channel.updateConnectionLayout(userId, deviceId, clientId, connectionId, layoutConfig);

// Prior to registration, ensure you create a channel claim with the CanUpdateLayout property set to true.
let theClaim: FMLiveSwitchChannelClaim = FMLiveSwitchChannelClaim.init()
theClaim.setId(_channelId)
theClaim.setCanUpdateLayout(true);
let channelClaims: NSMutableArray = []
claims.add(theClaim)

// Create a registration token with your ChannelClaims then call `_client?.register(withToken: theToken)`.

// Create a new FMLiveSwitchChannelConnectionLayoutConfig, and set the Zone and Priority as desired
let layoutConfig = FMLiveSwitchChannelConnectionLayoutConfig.init();
layoutConfig?.setZone(FMLiveSwitchRtmpLayoutZones.promotedLeft());
layoutConfig?.setPriority(FMLiveSwitchNullableInt.fromValue(0));

// Send the update - the _channel object here is returned when calling `_client?.register(withToken: theToken)`.
// The userId, deviceId, clientId, and connectionId must match a connection in the channel.
_channel?.updateConnectionLayout(withUserId: userId, deviceId: deviceId, clientId: clientId, connectionId: connectionId, newConfig: layoutConfig);

// Prior to registration, ensure you create a claim with the CanUpdateLayout property set to true.
const channelClaims = [new fm.liveswitch.ChannelClaim(channelId)];
channelClaims[0].setCanUpdateLayout(true);

// Create a registration token with your ChannelClaims then call `client.register(token)`.

// Create a new ChannelConnectionLayoutConfig, and set the Zone and Priority as desired
const layoutConfig = new fm.liveswitch.ChannelConnectionLayoutConfig();
layoutConfig.setZone(fm.liveswitch.RtmpLayoutZones.getPromotedLeft());
layoutConfig.setPriority(0);

// Send the update - the channel object here is returned when calling `client.register(token)`.
// The userId, deviceId, clientId, and connectionId must match a connection in the channel.
channel.updateConnectionLayout(userId, deviceId, clientId, connectionId, layoutConfig);

Function of Labeling Streams as Full or Promoted

The available RTMP layout zones are defined in FM.LiveSwitch.RtmpLayoutZones.

RtmpLayoutZones.Full: The connection uses all of the available space in the MCU output. If more than one connection is assigned to this zone, the connections will be arranged in a grid.
RtmpLayoutZones.FullSingle: The connection uses all of the available space in the MCU output. If more than one connection is assigned to this zone, only the connection with the lowest priority will be displayed.
RtmpLayoutZones.PromotedFloat: The connection floats above others and is aligned to the bottom-right corner of the MCU output. If more than one connection is assigned to this zone, only the connection with the lowest priority will be displayed. This zone is useful with the Full or FullSingle zones as one connection can be floated above a full screen connection.
RtmpLayoutZones.PromotedLeftSingle: The connection is displayed at 75% width of the MCU output and is aligned to the left edge. If multiple connections are assigned to this zone, the connection with the lowest priority will be displayed. All other connections will be displayed along the right edge in a single column.
RtmpLayoutZones.PromotedLeft: The connection is displayed at 75% width of the MCU output and is aligned to the left edge. If multiple connections are assigned to this zone, the connections will be arranged in a grid. All other connections will be displayed along the right edge in a single column.
RtmpLayoutZones.PromotedRight: The connection is displayed at 75% width of the MCU output and is aligned to the right edge. If multiple connections are assigned to this zone, the connections will be arranged in a grid. All other connections will be displayed along the left edge in a single column.
RtmpLayoutZones.PromotedTop: The connection is displayed at 75% height of the MCU output and is aligned to the top edge. If multiple connections are assigned to this zone, the connections will be arranged in a grid. All other connections will be displayed along the bottom edge in a single row.
RtmpLayoutZones.PromotedBottom: The connection is displayed at 75% height of the MCU output and is aligned to the bottom edge. If multiple connections are assigned to this zone, the connections will be arranged in a grid. All other connections will be displayed along the top edge in a single row.

Impact of Setting a Priority

Connections are given a priority as a number. Connections are ordered within zones by priority. Connections with a null priority value will be ordered last. Connections with the same priority will be ordered by join time.

Impact of Labeling More Than 5 Streams

Maximum number of displayed upstream connections is 5 regardless of zones assigned.

If 10 upstream connections are in the PromotedLeft zone, and there are total 15 upstream connections, then 5 upstream connections will be displayed in the PromotedLeft zone and there will be no visible connections on the right edge of the MCU output.

Zones are ordered as follows:

PromotedFloat
FullSingle
Full
PromotedLeftSingle
PromotedLeft
PromotedTop
PromotedRight
PromotedBottom

This allows the PromotedFloat zone to always render above all other zones, and allows the Full views to always take precedence over any promoted view.

For example, if one connection is assigned to the Full zone and another connection is PromotedLeft, the overall output will be a single connection in the Full zone.