DNN-Aided Message Passing Based Block Sparse Bayesian Learning for Joint User Activity Detection and Channel Estimation

by   Zhaoji Zhang, et al.

Faced with the massive connection, sporadic transmission, and small-sized data packets in future cellular communication, a grant-free non-orthogonal random access (NORA) system is considered in this paper, which could reduce the access delay and support more devices. In order to address the joint user activity detection (UAD) and channel estimation (CE) problem in the grant-free NORA system, we propose a deep neural network-aided message passing-based block sparse Bayesian learning (DNN-MP-BSBL) algorithm. In this algorithm, the message passing process is transferred from a factor graph to a deep neural network (DNN). Weights are imposed on the messages in the DNN and trained to minimize the estimation error. It is shown that the weights could alleviate the convergence problem of the MP-BSBL algorithm. Simulation results show that the proposed DNN-MP-BSBL algorithm could improve the UAD and CE accuracy with a smaller number of iterations.


DNN-Aided Block Sparse Bayesian Learning for User Activity Detection and Channel Estimation in Grant-Free Non-Orthogonal Random Access

In the upcoming Internet-of-Things (IoT) era, the communication is often...

Fixed-Symbol Aided Random Access Scheme for Machine-to-Machine Communications

The massiveness of devices in crowded Machine-to-Machine (M2M) communica...

Double-Sided Information Aided Temporal-Correlated Massive Access

This letter considers temporal-correlated massive access, where each dev...

An Enhanced SCMA Detector Enabled by Deep Neural Network

In this paper, we propose a learning approach for sparse code multiple a...

Study of Joint Activity Detection and Channel Estimation Based on Message Passing with RBP Scheduling for MTC

In this work, based on the hybrid generalized approximate message passin...

Deep-Learned Approximate Message Passing for Asynchronous Massive Connectivity

This paper considers the massive connectivity problem in an asynchronous...

Data-aided Active User Detection with a User Activity Extraction Network for Grant-free SCMA Systems

In grant-free sparse code multiple access system, joint optimization of ...

I Introduction

Providing efficient support for the Internet of Things (IoT) is one of the major objectives for the cellular wireless communication. Machine-to-Machine (M2M) communication is anticipated to support billions of Machine Type Communication (MTC) devices. In addition, the MTC devices are sporadically activated with short packets [1]. Therefore, the random access (RA) process for M2M communications in IoT is characterized by massive connection and sporadic transmission, as well as small-sized data packets.

Confronted with the characteristics described above, the conventional orthogonal multiple access (OMA) scheme becomes infeasible due to its low resource efficiency. To facilitate the sharing of uplink resources, different RA schemes were proposed and can be generally categorized into two types: grant-based RA [2, 3, 4] and grant-free RA [5, 6, 7, 8, 9, 10, 11, 12].

I-a Grant-Based Random Access

In grant-based RA schemes, activated users contend for the RBs by transmitting a preamble sequence to the base station (BS), while a RB is assigned to the activated user, whose preamble sequence is received and accepted by the BS. One problem with the grant-based RA schemes is that the RB is wasted when more than one active device transmit the same preamble sequence. Some solutions were proposed to alleviate the RA congestion by reducing the collision probability, such as the Access Class Barring (ACB) scheme

[2], delicate splitting of the RA preamble set [3], and automatic configuration of the RA parameters [4]. However, the RB wastage cannot be fully avoided by grant-based RA schemes, which results in low resource efficiency. Furthermore, a handshaking process is always to recognize the contention winner, which undermines the uplink transmission efficiency of small data packets.

I-B Grant-Free Random Access

I-B1 Compressed sensing-based grant-free RA schemes

Compressed sensing (CS) algorithms employ pilot sequences to accomplish the user activity detection (UAD) and/or channel estimation (CE) problem. For example, the joint UAD and CE problem was addressed by a modified Bayesian compressed sensing algorithm [5] for the cloud radio access network (C-RAN). In addition, the powerful approximate message passing (AMP) algorithm was employed for the joint UAD and CE problem when the BS is equipped either with a single antenna [6, 7] or with multiple antennas [8].

I-B2 Sparse Bayesian learning-based grant-free RA schemes

The sparse Bayesian learning (SBL) algorithm considers the prior hyper-parameter of the sparse signal. The Expectation Maximization (EM) method was employed by the AMP-SBL algorithm

[9] to update the sparse signal and the hyper-parameter. A least square (LS)-based AMP-SBL (LS-AMP-SBL) algorithm [10] was proposed to recover the sparse signal in three steps. Recently, a message-passing receiver design was proposed for the joint channel estimation and data decoding in uplink grant-free SCMA systems [11]. In addition, a message passing-based block sparse Bayesian learning (MP-BSBL) algorithm [12] was proposed for a grant-free NOMA system.

Fig. 1: LDS-OFDM based grant-free NORA system model. The joint UAD and CE problem is sloved by the proposed DNN-MP-BSBL algorithm, which processes the first received signals of the pilot sequences, while the MUD module is employed to process the remaining received signals to detect the data transmitted from active users.

I-C Contributions

In this paper, we consider a LDS-OFDM system, where devices perform grant-free RA once they are activated. A deep neural network-aided message passing-based block sparse Bayesian learning (DNN-MP-BSBL) algorithm is proposed to perform joint UAD and CE. The iterative message passing process of the MP-BSBL algorithm [12] is transferred from a factor graph to a neural network. Weights are imposed on the messages passing in the neural network and trained to minimize the estimation error.

The rest of this paper is organized as follows. The system model and the MP-BSBL algorithm are presented in Section II. The DNN structure for the DNN-MP-BSBL algorithm is illustrated in Section III, where the weighted message passing is explained in details. Simulation results are given in Section IV to verify the UAD and CE accuracy of the proposed DNN-MP-BSBL algorithm. Finally, Section V concludes this paper.

Ii Joint UAD and CE by MP-BSBL

Ii-a System Model and Problem Formulation

A LDS-OFDM system is considered in Fig. 1. There are sub-carriers and users. Each user is activated with probability . For active user , its information sequence is encoded and mapped into a QAM sequence with length . ZC sequences are adopted as pilot sequences. One unique sequence with length is allocated for user , and inserted into the transmitted sequence , i.e., . Therefore, the length of is . The LDS spreader for user is characterized by

, which is a sparse vector with length

and non-zero elements. The LDS spreaders for all the users are characterized by a LDS spreading matrix . We consider a regular , i.e., the column degree and the row degree in are constant. Each sub-carrier is shared by potential users, with . When multiple users are active on the same sub-carrier, the RA is conducted in a NOMA manner. After the OFDM de-modulator, the received matrix is


where the -th entry of represents the -th received symbol on the -th sub-carrier, the -th entry of represents the -th transmitted symbol of the -th user, and is a row sparse channel matrix, which integrates the activity of the users. The activity indicator if user is inactive. Otherwise, and the -th row of represents the channel gain vector on sub-carriers for user . The entries in the AWGN matrix

are assumed i.i.d with noise variance

. can be decomposed as , where and represent the received matrices w.r.t. the pilot sequences and the data sequences, respectively. We consider to solve the joint UAD and CE problem,


where is assumed known to the receiver. Then we perform vectorization on the transpose of as in [12],


where , , and is the channel gain vector of user on sub-carriers. According to the LDS spreading matrix , the transmitted symbols of each user are only spread onto sub-carriers. Therefore, elements in are zero. We further simplify by eliminating the zeros. Accordingly, the columns in corresponding to the zeros in are also removed. Finally, we obtain the simplified version of (2) in equation () of (3). According to (3), the joint UAD and CE problem is equivalent to recovering .

Ii-B MP-BSBL Algorithm [12]

The recovery of can be addressed by the MP-BSBL algorithm [12].

Fig. 2: Factor graph for the message passing in the MP-BSBL algorithm [12].

For user , the distribution of is assumed conditioned on a hyper-parameter , i.e., . The hyper-parameter

is assumed to follow a Gamma distribution. The noise precision

is unknown at the receiver but assumed with a priori probability

. With these assumptions above, the joint a posterior probability is factorized as follows


where , , is the -th row of matrix in (3), , , and ,

represents the complex Gaussian distribution probability density function (pdf) of

with mean and variance , while represents the Gamma distribution pdf of with parameters and . The parameters and are usually assumed in the order of .

A factor graph is established for the MP-BSBL algorithm in Fig. 2, where , and denote , , , and . The extra variable is introduced and the constraint is represented by . Then, is a function of and , i.e., . The MP-BSBL algorithm performed on the factor graph in Fig. 2 is briefed as follows

Denote as the iteration index and as the product of all the incoming messages from to . Then, the variance and mean of are,


The variance and mean of are updated as


The variance and mean from to are


Then is updated by the MF message passing,


The variance and mean of are


Then, is updated by MF message passing,


If , user is detected as inactive. Otherwise, is the estimated channel gain.

Iii DNN-Aided MP-BSBL Algorithm

Fig. 3: DNN for the weighted message passing in DNN-MP-BSBL algorithm with () and layer organization.


Index Layer Input Layer Output Length
1 None


TABLE I: Input and Output for Each Layer of the DNN.

The factor graph in Fig. 2 is densely connected, which results in the correlation problem of the Gaussian messages [13, 14, 15]. To address this problem, we propose a DNN-MP-BSBL algorithm to imposes weights on the Gaussian message update and the MF message update. To facilitate the training, the message passing is transferred from the factor graph to a DNN in Fig. 3(a). Each iteration of the MP-BSBL algorithm is now represented by one iteration block. Within each iteration block, one layer represents one particular message. Two auxiliary layers and are also added for illustration clarity. Therefore, as listed in Table I, there are 9 layers in each iteration block. The layer organization is shown in Fig. 3(b). The weighted message passing is represented by a weighting matrix , whose -th entry is non-zero if the -th input node is connected to the -th output node.

Iii-a Weighted Message Passing

Layer 1: Layer 1 is the input within one iteration block.

Layer 2: Layer 2 is the auxiliary layer and the output is derived as follows,


The fraction and operations are performed elementwise while the operation is the matrix multiplication.

Layer 3: The output of Layer 3 is derived as follows,


Layer 4: The output of Layer 4 are derived as follows,


Layer 5: The output is derived as follows,


Layer 6: The output and of Layer 6 is derived as

1.0 Input: , , , , weighting matrices
Output: , the index set of active user IDX
Initialize: for  do
       1. Update and by (12), (13), and (14). 2. Update and by (15). 3. Update by (16). 4. Update and by (17) and (18). 5. Update and by (19). 6. Update the noise precision by (20).
return: IDX=find,     .
Algorithm 1 DNN-MP-BSBL algorithm

Layer 7: The output of Layer 7 is


Layer 8: The output of Layer 8 is


Layer 9: The output of Layer 9 is,


Iii-B Summary of the Proposed DNN-MP-BSBL Algorithm

Finally, the proposed DNN-MP-BSBL algorithm is summarized in Algorithm 1. The mean square error (MSE)

is employed as the loss function for the training period, while the Normalized MSE (NMSE)

is considered for the simulations in the testing period.

Iv Simulations

Parameters for the simulations are listed in Table II. We consider a crowded NORA system with low-latency requirement, i.e., . The NMSE performances of the LS-AMP-SBL esimator [10], the BOMP estimator (with known active user number) [12] and the GA-MMSE estimator (with known user activity) are also considered. The NMSE performance of GA-MMSE estimator serves as the lower bound.


Parameter Symbol Value
User number 110
Subcarrier number 8
Pilot length 11
Spreading factor 4
Activation probability for each user 0.1
UAD threshold 0.1
Size of training set
Size of test set
Size of mini-batch 200
Epoch number 20
Learning rate


TABLE II: Related Parameters for Simulations

The simulation results are shown in Fig. 4, in crowded NORA systems, both the MP-BSBL algorithm and the BOMP estimator diverge from the NMSE lower bound as SNR increases, and the LS-AMP-SBL algorithm fails to work even with 50 iterations. By contrast, the DNN-MP-BSBL algorithm could closely approach the lower bound within a wide range of SNR. Therefore, the DNN-MP-BSBL algorithm requires fewer iterations and provides better NMSE performance, indicating its advantages in crowded NORA system with low-latency requirement.

Fig. 4: NMSE performance with different SNR.

V Conclusions

A DNN-MP-BSBL algorithm was proposed in this paper for the joint UAD and CE problem in grant-free NORA systems. The iterative message passing process is transferred from a factor graph to a DNN, while weights are imposed on the messages and trained to improve the UAD and CE accuracy. Simulation results showed that the NMSE performance of the DNN-MP-BSBL algorithm could approach the lower bound in a feasible number of iterations, indicating its advantages for low-latency NORA systems.


This work was supported by the NSFC under grant 61750110529.


  • [1] 3GPP, “Service Requirements for Machine-Type Communications,” TS 22.368 V13.1.0, Dec. 2014.
  • [2] Z. Wang and V. W. S. Wong, “Optimal access class barring for stationary machine type communication devices with timing advance information,” IEEE Trans. Wireless Commun., vol.14, no.10, pp.5374-5387, Oct. 2015.
  • [3] K.-D. Lee, S. Kim, and B. Yi, “Throughput comparison of random access methods for M2M service over LTE networks,” Proc. 2011 IEEE GLOBECOM Wkshps., Dec. 2011, pp. 373-377.
  • [4] S. Choi, W. Lee, D. Kim, K. J. Park, S. Choi, and K. Y. Han, “Automatic configuration of random access channel parameters in LTE systems,” Proc. 2011 IFIP Wireless Days, Oct. 2011, pp. 1-6.
  • [5] X. Xu, X. Rao, and V. K. N. Lau, “Active user detection and channel estimation in uplink CRAN systems,”IEEE Inter. Conf. Commun. (ICC), June 2015, pp. 2727-2732.
  • [6] G. Hannak, M. Mayer, A. Jung, G. Matz, and N. Goertz, “Joint channel estimation and activity detection for multiuser communication systems,” IEEE Inter. Conf. Commun. (ICC) Workshop, June 2015, pp. 2086-2091.
  • [7] Z. Chen and W. Yu, “Massive device activity detection by approximate message passing,” in IEEE Inter. Conf. Acoustics, Speech, Signal Processing (ICASSP), Mar. 2017, pp. 3514-3518.
  • [8] L. Liu and W. Yu, “Massive connectivity with massive MIMO-part I: device activity detection and channel estimation,” IEEE Transactions on Signal Processing, vol. 66, no. 11, pp. 2933-2946.
  • [9] M. Al-Shoukairi and B. Rao, “Sparse Bayesian learning using approximate message passing,” 48th Asilomar Conference on Signals, Systems and Computers, 2014, pp. 1957-1961.
  • [10] C. Wei, H. Liu, Z. Zhang, J. Dang, and L. Wu, “Near-optimum sparse channel estimation based on least squares and approximate message passing,” IEEE Wireless Communications Letters, vol. 6, no. 6, pp. 754-757.
  • [11] F. Wei, W. Chen, Y. Wu, J. Ma, and T. A. Tsiftsis, “Message-passing receiver design for joint channel estimation and data decoding in uplink grant-free SCMA systems,” IEEE Trans. Wireless Commun., vol. 18, no. 1, pp. 167-181, Jan. 2019.
  • [12] Y. Zhang, Q. Guo, Z. Wang, J. Xi, and N. Wu, “Block sparse Bayesian learning based joint user activity detection and channel estimation for grant-free NOMA systems,” IEEE Transactions on Vehicular Technology, vol. 67, no. 10, pp. 9631-9640.
  • [13] L. Liu, C. Yuen, Y. L. Guan, Y. Li, and Y. Su, “Convergence analysis and assurance for Gaussian message passing iterative detector in massive MU-MIMO systems,” IEEE Trans. Wireless Commun., vol. 15, no. 9, pp. 6487-6500.
  • [14] C. Huang, L. Liu, C. Yuen and S. Sun, “Iterative channel estimation using LSE and sparse message passing for MmWave MIMO systems,” IEEE Transactions on Signal Processing, vol. 67, no. 1, pp. 245-259, 1 Jan.1, 2019.
  • [15] C. Huang, L. Liu, and C. Yuen, “Asymptotically optimal estimation algorithm for the sparse signal with arbitrary distributions,” IEEE Transactions on Vehicular Technology, vol. 67, no. 10, pp. 10070-10075, Oct. 2018.