Intra-stream synchronization control is required for preservation of the timing relation between media units (MUs), each of which is an information unit for media synchronization, such as video frames in a single media stream.
The input intervals between MUs are disturbed owing to network delay jitter. The intra-stream synchronization control recovers the intervals when the destination outputs the MUs.

Intra-stream synchronization

Inter-stream synchronization control is required for keeping the temporal relationship among media streams.
Lip-sync is a representative of the inter-stream synchronization, and it means the synchronization between spoken voice and the movement of the speaker's lips.
Media streams fall into a master stream and slave streams. The slave streams are synchronized with the master stream. In lip-sync, the voice is generally selected as the master stream. This is because voice is more sensitive to intra-stream synchronization error than video.

Inter-stream synchronization

Group (or inter-destination) synchronization control is necessary to output each MU simultaneously at different destinations in multicast communications as shown in this slide. When the group synchronization is disturbed, the fairness among multiple users may be damaged.

Group synchronization

A number of media synchronization algorithms have been proposed to meet diverse requirements.
We group media synchronization control techniques used in the algorithms into the following four categories (excluding group synchronization control):

(1) Basic control

The basic control techniques are needed in almost all the algorithms, and they are indispensable to preserve the temporal relationships among media streams (e.g., attachment of synchronization information to MUs and buffering of MUs).

(2) Preventive control

The preventive control techniques are required to try to avoid asynchrony (i.e., out of synchronization). Thus, the techniques are used before asynchrony occurs (e.g., transmission of MUs according to synchronization information and change of buffering time with network delay estimation).

(3) Reactive control

The reactive control techniques are employed to recover from asynchrony after it has occurred (e.g., skipping and pausing MUs, shortening and extension of MU output duration, virtual-time contraction and expansion, and master-slave switching).

(4) Common control

The common control techniques can be used as both preventive and reactive control ones (adjustment of input/output rate and interpolation of data).

For example, the virtual-time rendering (VTR) media synchronization algorithm employs the virtual-time expansion and contraction, the shortening and extension of MU output duration, skipping and pausing of MUs, the attachment of synchronization information (timestamps) to MUs, the buffering of MUs, and so on.