An Efficient Deep Learning Model for Automatic Modulation Recognition Based on Parameter Estimation and Transformation

10/11/2021
by   Fuxin Zhang, et al.
0

Automatic modulation recognition (AMR) is a promising technology for intelligent communication receivers to detect signal modulation schemes. Recently, the emerging deep learning (DL) research has facilitated high-performance DL-AMR approaches. However, most DL-AMR models only focus on recognition accuracy, leading to huge model sizes and high computational complexity, while some lightweight and low-complexity models struggle to meet the accuracy requirements. This letter proposes an efficient DL-AMR model based on phase parameter estimation and transformation, with convolutional neural network (CNN) and gated recurrent unit (GRU) as the feature extraction layers, which can achieve high recognition accuracy equivalent to the existing state-of-the-art models but reduces more than a third of the volume of their parameters. Meanwhile, our model is more competitive in training time and test time than the benchmark models with similar recognition accuracy. Moreover, we further propose to compress our model by pruning, which maintains the recognition accuracy higher than 90 parameters comparing with state-of-the-art models.

READ FULL TEXT VIEW PDF

page 1

page 2

page 3

page 4

10/04/2018

Polar Feature Based Deep Architectures for Automatic Modulation Classification Considering Channel Fading

To develop intelligent receivers, automatic modulation classification (A...
12/01/2017

Deep Neural Network Architectures for Modulation Classification

In this work, we investigate the value of employing deep learning for th...
10/06/2021

SWAT Watershed Model Calibration using Deep Learning

Watershed models such as the Soil and Water Assessment Tool (SWAT) consi...
12/09/2019

DeepMUSIC: Direction Finding via Deep Learning

This letter introduces a deep learning (DL) framework for direction-of-a...
01/21/2021

Superiorities of Deep Extreme Learning Machines against Convolutional Neural Networks

Deep Learning (DL) is a machine learning procedure for artificial intell...
11/09/2020

Efficient Training Data Generation for Phase-Based DOA Estimation

Deep learning (DL) based direction of arrival (DOA) estimation is an act...
07/28/2022

Combining human parsing with analytical feature extraction and ranking schemes for high-generalization person reidentification

Person reidentification (re-ID) has been receiving increasing attention ...

Code Repositories

PET-CGDNN

An Efficient Deep Learning Model for Automatic Modulation Recognition Based on Parameter Estimation and Transformation


view repo

I Introduction

Automatic modulation recognition (AMR) has made it possible to identify signal modulation schemes automatically by the receiver in the non-cooperative communications scenarios, which has various civilian and military applications, such as spectral interference detection, spectrum sensing, cognitive radio, etc [1]. Deep learning (DL) based AMR methods are showing improved performance in terms of recognition accuracy and complexity compared with traditional likelihood-based methods and feature-based methods [6].

Prior DL-AMR models have achieved benchmark performance for AMR by using various neural network layers, e.g. convolutional neural network (CNN) [6, 2, 14]

, recurrent neural network (RNN)

[9, 3], and hybrid network [13], etc. Considering simple neural networks lacking the ability to eliminate signal distortion caused by the wireless channel, [15] designed a signal distortion correction module to equalize the carrier frequency and phase offset, which demonstrated the potential to improve DL-AMR models by incorporating expert domain knowledge. With the rapid development of 5G/B5G in recent years, the growth of massive Internet-of-things (IoT) devices demand improved communications performance with limited available resources [10], and thus efficient AMR models are crucially important for the future IoT devices with limited computing and energy resources. Consequently, researchers begin to research the lightweight and low-complexity DL-AMR models by reducing the model size or accelerating the computation time while ensuring the recognition accuracy [12, 4, 11], making them increasingly possible to deploy in resource-limited devices. However, the existing models with high recognition accuracy rarely considered the model size and complexity in the design process, while lightweight and low-complexity models struggle to achieve high accuracy.

In this letter, we propose an efficient DL-AMR model inspired by radio transformer networks (RTN)

[5], CNN, and GRU. The original data is processed by a parameter estimation network and a transformation module, and then the spatial and temporal features of the signals are extracted by CNN and gated recurrent unit (GRU) for classification. The model can achieve high recognition accuracy equivalent to state-of-the-art models but with much fewer parameters, while the training time and test time of our model outperform the benchmark models without suffering recognition accuracy. A network pruning method [16] is further applied to compress the model size while maintaining high recognition accuracy, making it a promising candidate for resource-limited systems.

The contributions of this letter are summarized as follows:

  • An efficient model that can achieve state-of-the-art recognition accuracy over three benchmark datasets but with least parameters is proposed based on parameter estimator, parameter transformer, CNN, and GRU.

  • To efficiently utilize the spatial-temporal features of AMR signals, we propose to decrease the kernel size and feature maps in the CNN layers, and introduce parameter estimator and transformer to reduce the adverse effects on phase, leading to improved recognition accuracy.

  • We demonstrate that the proposed lightweight model can be further compressed by five times using pruning method to fit the scenarios with extremely limited resources.

Ii Signal Model and Proposed System Model

Ii-a Signal Model

After the signal passes through the channel and is sampled, the equivalent baseband signal can be expressed by:

(1)

where is the signal modulated by the transmitter in a certain modulation scheme, denotes the complex Additive Gaussian Noise (AWGN), represents the channel gain, is the frequency offset, is the phase offset, denotes the -th value observed by the receiver and is the number of symbols in a signal sample. To facilitate data processing and modulation recognition, the received signals can be stored in in-phase/quadrature (I/Q) form, denoted as .

Ii-B The Proposed DL Model

The proposed parameter estimation and transformation based CNN-GRU deep neural network (PET-CGDNN) model comprises a parameter estimator, a parameter transformer, and a hybrid neural network, which is shown in Fig. 1.

Fig. 1: The structure of the proposed PET-CGDNN.

As described in the signal model, the sampled I/Q data is affected by noise and interference from the channel and imperfect hardware design, which may result in adverse effects such as temporal shifting, linear mixing/rotating, and spinning of the received signal. Many of such effects can be inverted using parametric transformations according to classic signal processing theories. Hence, the parameter estimator (Part 1) and the parameter transformer (Part 2) have been introduced in our model to extract phase offset related information and perform phase parameter transformation, which are the key to enhancing the recognition accuracy of our model.

The parameter estimator in Part 1 can estimate the phase parameter by co-training with the subsequent model. For each I/Q sample with the dimension of 2 × ( = 128 or 1024), one phase parameter, which carries the phase offset related information, is estimated by a neural network composed of a Flatten layer and a Dense layer (fully connected layer). The input signal y

is flattened into a vector by the Flatten layer to satisfy the input dimension of the Dense layer, and then the data in this vector are correlated through the Dense layer to obtain the phase parameter

. The activation function of the Dense layer is Linear, which yields an estimated phase parameter from a continuous unbounded range.

The parameter transformer in Part 2 is a customized layer, which performs parametric inverse transformation by taking the input y and , given by:

(2)

where is the estimated phase parameter and is the output of Part 2.

Part 3 consists of CNN, GRU, and Dense layer which can realize feature extraction and classification. The first convolutional layer has 75 filters and 2 × 8 kernel size, which extracts the spatial features of the signal, while the second convolutional layer has 25 filters and 1 × 5 kernel size, which further compacts the extracted features. The subsequent GRU layer extracts the temporal features of the signal with 128 units. Finally, the classification task is completed through the Dense layer, and the number of hidden units is C, which is equal to the number of modulation classes. The first two convolution layers use rectified linear unit (ReLU) activation functions, and the activation function of the last Dense layer is Softmax. Aiming at building a model with a smaller size and low computational cost, the corresponding network in the third part is designed to control the model size with a small number of parameters in the CNN layer.

Ii-C Model Pruning Method

Although our model already has a small number of parameters, CNN based structures usually have intrinsic redundancy, which could cause a large model size and unnecessary computational cost. To further compress the model, a network pruning method is adopted to reduce the redundancy. We apply the following approach for the proposed DL-AMR model, aiming to maintain a high recognition accuracy while reducing the model size, modeled as follows,

(3)

where is initial sparsity value, denotes final sparsity value, refers to the step to start training with pruning frequency , the fine-tuning process is divided into steps to gradually increase the sparsity, and

. Binary mask variables of the same size and shape as the weights tensor are added to Dense, CNN, and GRU layers. The weights masks of each layer are sorted in the fine-tuning process, and the corresponding weights in the smallest portion are set to zero until reaching the target sparsity goal.

Iii Datasets and Implementation Details

The experiments are conducted on RML2016.10a, RML2016.10b [7] and RML2018.01a [8] datasets with the input data dimension of 2 × 128, 2 × 128 and 2 × 1024, respectively. RML2016.10a dataset includes 220,000 modulated signals with 11 commonly used modulation schemes, RML2016.10b dataset contains 1,200,000 signals with 10 schemes, and RML2018.01a dataset has over 2.5 million signals with 24 modulation schemes. Owing to the hardware limitations, only half of RML2018.01a dataset is randomly selected in our experiment. RML2016.10a and RML2016.10b are generated by simulation using GNU radio while RML2018.01a is produced in a laboratory environment.

We divide the datasets into training, validation and test at the ratio of 6:2:2 per class with random selection. The loss function is categorical cross-entropy, and the optimizer is Adam. When the validation loss does not decrease in 5 epochs, it is multiplied by a coefficient of 0.5. When the validation loss does not decrease in 50 epochs, the training process is stopped and the trained model is saved with the minimum validation loss. The experiments are implemented using GeForce GTX 1080Ti GPU and Keras with Tensorflow as the backend.

Key AMR models are implemented to provide benchmark comparison which includes IC-AMCNET [2], MCNET [4], LSTM2 [9], GRU2 [3], MCLDNN [13].

Iv Experimental Results and Discussion

Iv-a Model Performance Measurement

Several indicators are selected in Table I

for performance comparison and complexity analysis, including the number of parameters, training time, test time, highest accuracy of all SNRs (signal-to-noise ratio), and average accuracy of all SNRs. All models are assessed on the three datasets, with input and output layers adjusted to fit the data dimensions, so the number of parameters varies, as shown in the table. Our models in Table

I are the vanilla version without pruning.

Model Datasets Parameters
Training time
(second/epoch)
Test time
(ms/sample)
Highest
accuracy
Average
accuracy
IC-AMCNET
A
B
C
1,264,011
1,263,882
8,605,720
6
34
172
0.036
0.015
0.063
85.59%
92.82%
94.42%
56.83%
62.15%
58.4%
MCNET
A
B
C
121,511
121,226
126,616
8
46
100
0.041
0.017
0.053
83.91%
89.08%
91.21%
56.63%
60.95%
57.04%
LSTM2
A
B
C
201,099
200,970
202,776
11
56
497
0.047
0.032
0.239
91.41%
94.01%
98.39%
60.56%
63.28%
65.49%
GRU2
A
B
C
151,179
151,050
152,856
9
48
313
0.043
0.026
0.151
87.86%
93.63%
98.24%
58.9%
63.7%
63.17%
MCLDNN
A
B
C
406,199
406,070
407,876
17
90
662
0.061
0.045
0.296
92.95%
93.86%
98.11%
62.08%
63.78%
62.59%
PET-CGDNN
(Ours)
A
B
C
71,871
71,742
75,340
6
33
208
0.039
0.02
0.099
91.36%
93.41%
98.16%
60.44%
63.82%
63.01%
TABLE I: Model comparison on three datasets (A: RML2016.10a, B: RML2016.10b, C: RML2018.01a).

It is clear that the proposed PET-CGDNN model without pruning is already with the least parameters compared with benchmark models in Table I. The time cost of PET-CGDNN has obvious advantages compared with GRU2, LSTM2, and MCLDNN, and is comparable with IC-ACMNET and MCNET but having much higher accuracy.

Note that the parameters of PET-CGDNN are relatively stable when tested on datasets with a larger input dimension. The IC-AMCNET, originally designed for the dataset with an input data length of 128, would see 6.8 times more parameters for higher-dimensional input data (RML2018.01a). In addition, IC-AMCNET and MCNET are composed of CNNs, which have lower computational complexity than RNN, leading to a low level of training and test time. PET-CGDNN has some disadvantages in time cost compared with IC-AMCNET and MCNET, but the relatively higher cost is offset by the higher recognition accuracy in all test cases. Specifically, PET-CGDNN has the shortest training time on RML2016.10b. Comparing with benchmark high accuracy models such as MCLDNN and LSTM, our model sacrifices little recognition accuracy but greatly reduces the complexity and model size.

(a)
(b)
(c)
(d)
Fig. 2: Recognition accuracy comparison on different datasets: (a) RML2016.10a, (b) RML2016.10b, (c) RML2018.01a, (d) ablation experiment comparison between PET-CGDNN and PET-CGDNN-Part 3 in Section IV-C (A: RML2016.10a, B: RML2016.10b, C: RML2018.01a).

Fig. 2LABEL:sub@fig16a to Fig. 2LABEL:sub@fig18 provide detailed recognition accuracy against SNR on the three datasets. These figures demonstrate that our PET-CGDNN model maintains stable and consistent performance on all three datasets in comparison with the benchmark high accuracy models.

Iv-B Model Pruning

We implement the pruning method in TensorFlow, which has been integrated into TensorFlow as a Keras-based pruning tool. In our experiment, the fine-tuning process has 5 epochs which are divided into 5,160 steps by setting the batch size to 128. The model is pruned and tested with different sparsity, while the size of the model is measured by the number of non-zero (NNZ) parameters. Table II and Fig. 3 present the number of model parameters and the recognition accuracy after pruning. While the number of parameters is less than 15K with a sparsity of 0.8, the pruned model can maintain an accuracy above 90%. The recognition accuracy of the pruned models remains stable on RML2016.10a and RML2016.10b when the sparsity is between 0 and 0.8, even though the model size is only 1/5 of the original model. The recognition accuracy of the pruned models decreases slightly on RML2018.01a, which indicates more connections are needed to fully extract information from the data with an input length of 1024.

Sparsity Datasets
NNZ
parameters
Highest
accuracy
Average
accuracy
0 (Original)
A
B
C
71K
71K
75K
91.36%
93.41%
98.16%
60.44%
63.82%
63.01%
0.5
A
B
C
35K
35K
37K
91.09%
93.28%
94.86%
60.43%
63.68%
59.49%
0.8
A
B
C
14K
14K
15K
90.36%
93.26%
93.19%
59.87%
63.56%
57.99%
0.9
A
B
C
7K
7K
7.4K
87.91%
92.78%
92.74%
55.61%
62.59%
57.21%
0.95
A
B
C
3.6K
3.6K
3.7K
81.18%
91.18%
83.14%
49.53%
61.17%
51.06%
TABLE II: Model performance after pruning (A: RML2016.10a, B: RML2016.10b, C: RML2018.01a).
(a)
(b)
Fig. 3: Model recognition accuracy after pruning: (a) RML2016.10a, (b) RML2018.01a.

Iv-C Effectiveness Analysis of PET-CGDNN

To further illustrate the effectiveness of the proposed model, we conduct an ablation experiment to compare the recognition accuracy of PET-CGDNN with the final module: PET-CGDNN-Part 3, which reflects the contribution of Part 1 and Part 2 to the recognition accuracy. It can be seen from Fig. 2LABEL:sub@figcom that the model (PET-CGDNN-Part 3) without the parameter estimation and transformation module cannot achieve the equivalent recognition accuracy of PET-CGDNN in the high SNR range (above 0 dB). The average recognition accuracy and the best recognition accuracy of PET-CGDNN exhibit overall better performance, while the model size and time cost are almost the same.

Fig. 4: The output of Part 2 and its comparison with the original input.

To further assess the functions of the parameter estimator and parameter transformer (Part 1 and Part 2) in PET-CGDNN, the output of Part 2 is visualized in the I/Q plane with a focus on the constellation distribution features rather than the value of these points (the origin of coordinates are different in each image). Fig. 4 shows the output of Part 2 in comparison with the inputs to the model when the signals’ SNR is +10 dB. It is clearly visible that the signals converge to tighter clusters after the two modules, which benefit the classification module and lead to the improvement of the overall recognition accuracy compared with PET-CGDNN-Part 3.

V Conclusion

In this letter, an efficient DL-AMR model, named PET-CGDNN, is proposed to achieve state-of-the-art performance. With the blessing of expert domain knowledge on phase offset estimation and compensation, the model enjoys the characteristics of lightweight, low-complexity, and high recognition accuracy, which also exhibits good stability on different datasets. Moreover, a pruning method has been applied to further reduce the model size, which exhibits the possibility to compress an AMR model, even if the model parameters are already very small. The efficient AMR model will have potentially wide application in the scenarios of massive machine-type communications and ultra-reliable and low latency communications in the future, which is in line with the development trend of future communication systems.

References

  • [1] O. A. Dobre, A. Abdi, Y. Bar-Ness, and W. Su (2007-Apr.) Survey of automatic modulation classification techniques: Classical approaches and new trends. IET commun. 1 (2), pp. 137–156. Cited by: §I.
  • [2] A. P. Hermawan, R. R. Ginanjar, D. Kim, and J. Lee (2020-Jan.) CNN-based automatic modulation classification for beyond 5G communications. IEEE Commun. Lett. 24 (5), pp. 1038–1041. Cited by: §I, §III.
  • [3] D. Hong, Z. Zhang, and X. Xu (2017-Dec.) Automatic modulation classification using recurrent neural networks. In Proc. IEEE Int. Conf. Comput. Commun., pp. 695–700. Cited by: §I, §III.
  • [4] T. Huynh-The, C. Hua, Q. Pham, and D. Kim (2020-Jan.) MCNet: An efficient CNN architecture for robust automatic modulation classification. IEEE Commun. Lett. 24 (4), pp. 811–815. Cited by: §I, §III.
  • [5] T. O’shea and J. Hoydis (2017-Oct.) An introduction to deep learning for the physical layer. IEEE Trans. Cogn. Commun. Netw. 3 (4), pp. 563–575. Cited by: §I.
  • [6] T. J. O’Shea, J. Corgan, and T. C. Clancy (2016) Convolutional radio modulation recognition networks. In Proc. Int. Conf. Eng. Appl. Neural Netw., pp. 213–226. Cited by: §I, §I.
  • [7] T. J. O’shea and N. West (2016)

    Radio machine learning dataset generation with gnu radio

    .
    In Proc. GNU Radio Conf., Vol. 1. Cited by: §III.
  • [8] T. J. O’Shea, T. Roy, and T. C. Clancy (2018-Jan.) Over-the-air deep learning based radio signal classification. IEEE J. Sel. Top. Signal Process. 12 (1), pp. 168–179. Cited by: §III.
  • [9] S. Rajendran, W. Meert, D. Giustiniano, V. Lenders, and S. Pollin (2018-05) Deep learning models for wireless signal classification with distributed low-cost spectrum sensors. IEEE Trans. Cogn. Commun. Netw. 4 (3), pp. 433–445. Cited by: §I, §III.
  • [10] M. Shafi, A. F. Molisch, P. J. Smith, T. Haustein, P. Zhu, P. De Silva, F. Tufvesson, A. Benjebbour, and G. Wunder (2017-Apr.) 5G: A tutorial overview of standards, trials, challenges, deployment, and practice. IEEE J. Sel. Areas Commun 35 (6), pp. 1201–1221. Cited by: §I.
  • [11] G. B. Tunze, T. Huynh-The, J. Lee, and D. Kim (2020-Dec.) Sparsely connected CNN for efficient automatic modulation recognition. IEEE Trans. Veh. Technol. 69 (12), pp. 15557–15568. Cited by: §I.
  • [12] Y. Wang, J. Yang, M. Liu, and G. Gui (2020-Feb.) LightAMC: Lightweight automatic modulation classification via deep learning and compressive sensing. IEEE Trans. Veh. Technol. 69 (3), pp. 3491–3495. Cited by: §I.
  • [13] J. Xu, C. Luo, G. Parr, and Y. Luo (2020-Jun.) A spatiotemporal multi-channel learning framework for automatic modulation recognition. IEEE Wireless Commun. Lett. 9 (10), pp. 1629–1632. Cited by: §I, §III.
  • [14] Z. Xu, I. Petrunin, and A. Tsourdos (2020-Jul.) Identification of communication signals using learning approaches for cognitive radio applications. IEEE Access 8, pp. 128930–128941. Cited by: §I.
  • [15] K. Yashashwi, A. Sethi, and P. Chaporkar (2018-Jul.) A learnable distortion correction module for modulation recognition. IEEE Wireless Commun. Lett. 8 (1), pp. 77–80. Cited by: §I.
  • [16] M. Zhu and S. Gupta (2017) To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01878. Cited by: §I.