Grace++: Loss-Resilient Real-Time Video Communication under High Network Latency

05/21/2023
by   Yihua Cheng, et al.
0

In real-time videos, resending any packets, especially in networks with high latency, can lead to stuttering, poor video quality, and user frustration. Despite extensive research, current real-time video systems still use redundancy to handle packet loss, thus compromising on quality in the the absence of packet loss. Since predicting packet loss is challenging, these systems only enhance their resilience to packet loss after it occurs, leaving some frames insufficiently protected against burst packet losses. They may also add too much redundancy even after the packet loss has subsided. We present Grace++, a new real-time video communication system. With Grace++, (i) a video frame can be decoded, as long as any non-empty subset of its packets are received, and (ii) the quality gracefully degrades as more packets are lost, and (iii) approximates that of a standard codec (like H.265) in absence of packet loss. To achieve this, Grace++ encodes and decodes frames by using neural networks (NNs). It uses a new packetization scheme that makes packet loss appear to have the same effect as randomly masking (zeroing) a subset of elements in the NN-encoded output, and the NN encoder and decoder are specially trained to achieve decent quality if a random subset of elements in the NN-encoded output are masked. Using various test videos and real network traces, we show that the quality of Grace++ is slightly lower than H.265 when no packets are lost, but significantly reduces the 95th percentile of frame delay (between encoding a frame and its decoding) by 2x when packet loss occurs compared to other loss-resilient schemes while achieving comparable quality. This is because Grace++ does not require retransmission of packets (unless all packets are lost) or skipping of frames.

READ FULL TEXT

page 6

page 18

research
10/29/2022

GRACE: Loss-Resilient Real-Time Video Communication Using Data-Scalable Autoencoder

Across many real-time video applications, we see a growing need (especia...
research
05/23/2023

Reparo: Loss-Resilient Generative Codec for Video Conferencing

Loss of packets in video conferencing often results in poor quality and ...
research
07/07/2021

Adversarial Auto-Encoding for Packet Loss Concealment

Communication technologies like voice over IP operate under constrained ...
research
07/03/2022

Towards Error-Resilient Neural Speech Coding

Neural audio coding has shown very promising results recently in the lit...
research
02/09/2022

Improving Content-Aware Video Streaming in Congested Networks with In-Network Computing

Network congestion and packet loss pose an ever-increasing challenge to ...
research
11/19/2019

Packet Loss Recovery in Broadcast for Real-Time Applications in Dense Wireless Networks

Packet loss recovery in wireless broadcast is challenging, particularly ...
research
10/28/2019

The Effect of Erasure Coding on the Burstiness of Packet Loss

The perceived quality of real-time media delivered over IP networks depe...

Please sign up or login with your details

Forgot password? Click here to reset