Robust Semantic Communications with Masked VQ-VAE Enabled Codebook

06/08/2022
by   Qiyu Hu, et al.
0

Although semantic communications have exhibited satisfactory performance for a large number of tasks, the impact of semantic noise and the robustness of the systems have not been well investigated. Semantic noise refers to the misleading between the intended semantic symbols and received ones, thus cause the failure of tasks. In this paper, we first propose a framework for the robust end-to-end semantic communication systems to combat the semantic noise. In particular, we analyze sample-dependent and sample-independent semantic noise. To combat the semantic noise, the adversarial training with weight perturbation is developed to incorporate the samples with semantic noise in the training dataset. Then, we propose to mask a portion of the input, where the semantic noise appears frequently, and design the masked vector quantized-variational autoencoder (VQ-VAE) with the noise-related masking strategy. We use a discrete codebook shared by the transmitter and the receiver for encoded feature representation. To further improve the system robustness, we develop a feature importance module (FIM) to suppress the noise-related and task-unrelated features. Thus, the transmitter simply needs to transmit the indices of these important task-related features in the codebook. Simulation results show that the proposed method can be applied in many downstream tasks and significantly improve the robustness against semantic noise with remarkable reduction on the transmission overhead.

READ FULL TEXT

page 26

page 27

page 28

research
02/07/2022

Robust Semantic Communications Against Semantic Noise

Although the semantic communications have exhibited satisfactory perform...
research
07/22/2021

Semantic Communications for Speech Recognition

The traditional communications transmit all the source date represented ...
research
02/27/2023

Wireless End-to-End Image Transmission System using Semantic Communications

Semantic communication is considered the future of mobile communication,...
research
09/23/2022

Vector Quantized Semantic Communication System

Although analog semantic communication systems have received considerabl...
research
10/30/2022

Semantic-Native Communication: A Simplicial Complex Perspective

Semantic communication enables intelligent agents to extract meaning (or...
research
09/21/2022

Robust Information Bottleneck for Task-Oriented Communication with Digital Modulation

Task-oriented communications, mostly using learning-based joint source-c...
research
02/07/2022

Semantic-aware Speech to Text Transmission with Redundancy Removal

Deep learning (DL) based semantic communication methods have been explor...

Please sign up or login with your details

Forgot password? Click here to reset