Create Streams and Connections

Note

About the code examples on this page:

For .NET MAUI and Unity, use the C# code.
For macOS, use the iOS code.

Create Streams

You've learned that local media captures the audio and video data of the current user, and remote media displays the audio and video data of other users. What's needed now is some way to tie these together. The object that does this is known as a stream. A stream is a relationship between two media objects that defines how these objects communicate with each other. This section focuses on how to use the App.LocalMedia and App.RemoteMedia classes that you have created to establish this communication.

Create Bidirectional Streams

The most common type of stream is one that both sends and receives data. This is known as a bi-directional stream, because media data moves in two directions. To stream audio/video data, you must create two streams, one for audio data and one for video data. These streams are represented by the FM.LiveSwitch.AudioStream and FM.LiveSwitch.VideoStream classes. To create bi-directional audio and video streams, create an instance of each of these classes, and provide both your App.LocalMedia instance and an instance of your App.RemoteMedia class.

Note that you only ever have one instance of your App.LocalMedia class. You create this instance once, and you invoke Start on this instance once. You , however, have multiple instances of your App.RemoteMediaclass. You need one remote media instance for each participant that joins a session. In turn, because a stream defines a relationship between two points, you need one instance of AudioStream and one instance of VideoStream per user that joins a video conference.

The code sample below shows how you would establish a single connection with a user.

var remoteMedia = new RemoteMedia();
var audioStream = new FM.LiveSwitch.AudioStream(localMedia, remoteMedia);
var videoStream = new FM.LiveSwitch.VideoStream(localMedia, remoteMedia);

RemoteMedia remoteMedia = new RemoteMedia();
fm.liveswitch.AudioStream = new fm.liveswitch.AudioStream(localMedia, remoteMedia)
fm.liveswitch.VideoStream = new fm.liveswitch.VideoStream(localMedia, remoteMedia)

var remoteMedia = RemoteMedia()
var audioStream = FMLiveSwitchAudioStream(localMedia: localMedia, remoteMedia: remoteMedia)
var videoStream = FMLiveSwitchVideoStream(localMedia: localMedia, remoteMedia: remoteMedia)

var remoteMedia = new fm.liveswitch.RemoteMedia();
var audioStream = new fm.liveswitch.AudioStream(localMedia, remoteMedia);
var videoStream = new fm.liveswitch.VideoStream(localMedia, remoteMedia);

Create Unidirectional Streams

You can also create a uni-directional stream. This is a stream in which data is either only sent or only received. If you pass null instead of a LocalMedia instance to your audio or video stream's constructor, then no data is sent through the stream. This is known as a receive-only stream, demonstrated below:

var remoteMedia = new RemoteMedia();
var audioStream = new FM.LiveSwitch.AudioStream(null, remoteMedia);
var videoStream = new FM.LiveSwitch.VideoStream(null, remoteMedia);

RemoteMedia remoteMedia = new RemoteMedia();
fm.liveswitch.AudioStream audioStream = new fm.liveswitch.AudioStream(null, remoteMedia);
fm.liveswitch.VideoStream videoStream = new fm.liveswitch.VideoStream(null, remoteMedia);

var remoteMedia = RemoteMedia()
var audioStream = FMLiveSwitchAudioStream(localMedia: nil, remoteMedia: remoteMedia)
var videoStream = FMLiveSwitchVideoStream(localMedia: nil, remoteMedia: remoteMedia)

var remoteMedia = new fm.liveswitch.RemoteMedia();
var audioStream = new fm.liveswitch.AudioStream(null, remoteMedia);
var videoStream = new fm.liveswitch.VideoStream(null, remoteMedia);

The opposite of this is a send-only stream. In this case, you pass null instead of a RemoteMedia instance (see the example below). You now have local media and remote media, which control how data is captured and displayed. You also have streams that control how data is sent and received. What you need now is something to tie this all together. The final section talks about how to do this by creating a connection. We cover this next.

var audioStream = new FM.LiveSwitch.AudioStream(localMedia, null);
var videoStream = new FM.LiveSwitch.VideoStream(localMedia, null);

fm.liveswitch.AudioStream audioStream = new fm.liveswitch.AudioStream(localMedia, null);
fm.liveswitch.VideoStream videoStream = new fm.liveswitch.VideoStream(localMedia, null);

var audioStream = FMLiveSwitchAudioStream(localMedia: localMedia, remoteMedia: nil)
var videoStream = FMLiveSwitchVideoStream(localMedia: localMedia, remoteMedia: nil)

var audioStream = new fm.liveswitch.AudioStream(localMedia, null);
var videoStream = new fm.liveswitch.VideoStream(localMedia, null);

Create a Connection

A connection is an object that communicates between two endpoints using the ICE (Interactive Connectivity Establishment) protocol. ICE itself is beyond the scope of this guide. All that you need to know for now is that a connection is responsible for establishing and maintaining communication between two parties in a video conference.

LiveSwitch offers three types of connections: MCU, SFU, and P2P. These are covered in detail in subsequent sections. For now, let's demonstrate a simple example of Connection creation by creating an MCU Connection from a Channel.

At this point, you should have an FM.LiveSwitch.Channel instance that you obtained either during registration or by joining a channel afterwards. To create an McuConnection instance, invoke the CreateMcuConnection method of the channel, as shown below, and pass in your AudioStream and VideoStream instances.

var connection = channel.CreateMcuConnection(audioStream, videoStream);

fm.liveswitch.McuConnection = channel.createMcuConnection(audioStream, videoStream);

var connection = channel.createMcuConnection(audioStream: audioStream, videoStream: videoStream)

var connection = channel.createMcuConnection(audioStream, videoStream);

The same procedure works for peer connections (using CreatePeerConnection) and SFU connections (using CreateSfuUpstreamConnection or CreateSfuDownstreamConnection).

MCU connections and peer connections can be bi-directional (send/receive) or uni-directional (send-only or receive-only). SFU connections are always uni-directional. SFU upstream connections are always send-only (local media only) and SFU downstream connections are always receive-only (remote media only).

If Media-over-WebSockets protocol is requested, set the boolean to true in the method overload to indicate that media should be sent over WebSockets. Only supported in C# and JavaScript and only for SFU connections. Data Streams are not supported for Media-over-WebSockets:

CSharp
JavaScript

var upstreamConnection = channel.CreateSfuUpstreamConnection(audioStream, videoStream, true);
var downstreamConnection = channel.CreateSfuUpstreamConnection(audioStream, videoStream, true);

var upstreamConnection = channel.createSfuUpstreamConnection(audioStream, videoStream, true);
var downstreamConnection = channel.createSfuDownstreamConnection(audioStream, videoStream, true);

Refer to Media-over-WebSockets to find out more.