I Introduction
Providing efficient support for the Internet of Things (IoT) is one of the major objectives for the cellular wireless communication. MachinetoMachine (M2M) communication is anticipated to support billions of Machine Type Communication (MTC) devices. In addition, the MTC devices are sporadically activated with short packets [1]. Therefore, the random access (RA) process for M2M communications in IoT is characterized by massive connection and sporadic transmission, as well as smallsized data packets.
Confronted with the characteristics described above, the conventional orthogonal multiple access (OMA) scheme becomes infeasible due to its low resource efficiency. To facilitate the sharing of uplink resources, different RA schemes were proposed and can be generally categorized into two types: grantbased RA [2, 3, 4] and grantfree RA [5, 6, 7, 8, 9, 10, 11, 12].
Ia GrantBased Random Access
In grantbased RA schemes, activated users contend for the RBs by transmitting a preamble sequence to the base station (BS), while a RB is assigned to the activated user, whose preamble sequence is received and accepted by the BS. One problem with the grantbased RA schemes is that the RB is wasted when more than one active device transmit the same preamble sequence. Some solutions were proposed to alleviate the RA congestion by reducing the collision probability, such as the Access Class Barring (ACB) scheme
[2], delicate splitting of the RA preamble set [3], and automatic configuration of the RA parameters [4]. However, the RB wastage cannot be fully avoided by grantbased RA schemes, which results in low resource efficiency. Furthermore, a handshaking process is always to recognize the contention winner, which undermines the uplink transmission efficiency of small data packets.IB GrantFree Random Access
IB1 Compressed sensingbased grantfree RA schemes
Compressed sensing (CS) algorithms employ pilot sequences to accomplish the user activity detection (UAD) and/or channel estimation (CE) problem. For example, the joint UAD and CE problem was addressed by a modified Bayesian compressed sensing algorithm [5] for the cloud radio access network (CRAN). In addition, the powerful approximate message passing (AMP) algorithm was employed for the joint UAD and CE problem when the BS is equipped either with a single antenna [6, 7] or with multiple antennas [8].
IB2 Sparse Bayesian learningbased grantfree RA schemes
The sparse Bayesian learning (SBL) algorithm considers the prior hyperparameter of the sparse signal. The Expectation Maximization (EM) method was employed by the AMPSBL algorithm
[9] to update the sparse signal and the hyperparameter. A least square (LS)based AMPSBL (LSAMPSBL) algorithm [10] was proposed to recover the sparse signal in three steps. Recently, a messagepassing receiver design was proposed for the joint channel estimation and data decoding in uplink grantfree SCMA systems [11]. In addition, a message passingbased block sparse Bayesian learning (MPBSBL) algorithm [12] was proposed for a grantfree NOMA system.IC Contributions
In this paper, we consider a LDSOFDM system, where devices perform grantfree RA once they are activated. A deep neural networkaided message passingbased block sparse Bayesian learning (DNNMPBSBL) algorithm is proposed to perform joint UAD and CE. The iterative message passing process of the MPBSBL algorithm [12] is transferred from a factor graph to a neural network. Weights are imposed on the messages passing in the neural network and trained to minimize the estimation error.
The rest of this paper is organized as follows. The system model and the MPBSBL algorithm are presented in Section II. The DNN structure for the DNNMPBSBL algorithm is illustrated in Section III, where the weighted message passing is explained in details. Simulation results are given in Section IV to verify the UAD and CE accuracy of the proposed DNNMPBSBL algorithm. Finally, Section V concludes this paper.
Ii Joint UAD and CE by MPBSBL
Iia System Model and Problem Formulation
A LDSOFDM system is considered in Fig. 1. There are subcarriers and users. Each user is activated with probability . For active user , its information sequence is encoded and mapped into a QAM sequence with length . ZC sequences are adopted as pilot sequences. One unique sequence with length is allocated for user , and inserted into the transmitted sequence , i.e., . Therefore, the length of is . The LDS spreader for user is characterized by
, which is a sparse vector with length
and nonzero elements. The LDS spreaders for all the users are characterized by a LDS spreading matrix . We consider a regular , i.e., the column degree and the row degree in are constant. Each subcarrier is shared by potential users, with . When multiple users are active on the same subcarrier, the RA is conducted in a NOMA manner. After the OFDM demodulator, the received matrix is(1) 
where the th entry of represents the th received symbol on the th subcarrier, the th entry of represents the th transmitted symbol of the th user, and is a row sparse channel matrix, which integrates the activity of the users. The activity indicator if user is inactive. Otherwise, and the th row of represents the channel gain vector on subcarriers for user . The entries in the AWGN matrix
are assumed i.i.d with noise variance
. can be decomposed as , where and represent the received matrices w.r.t. the pilot sequences and the data sequences, respectively. We consider to solve the joint UAD and CE problem,(2) 
where is assumed known to the receiver. Then we perform vectorization on the transpose of as in [12],
(3) 
where , , and is the channel gain vector of user on subcarriers. According to the LDS spreading matrix , the transmitted symbols of each user are only spread onto subcarriers. Therefore, elements in are zero. We further simplify by eliminating the zeros. Accordingly, the columns in corresponding to the zeros in are also removed. Finally, we obtain the simplified version of (2) in equation () of (3). According to (3), the joint UAD and CE problem is equivalent to recovering .
IiB MPBSBL Algorithm [12]
The recovery of can be addressed by the MPBSBL algorithm [12].
For user , the distribution of is assumed conditioned on a hyperparameter , i.e., . The hyperparameter
is assumed to follow a Gamma distribution. The noise precision
is unknown at the receiver but assumed with a priori probability. With these assumptions above, the joint a posterior probability is factorized as follows
(4) 
where , , is the th row of matrix in (3), , , and ,
represents the complex Gaussian distribution probability density function (pdf) of
with mean and variance , while represents the Gamma distribution pdf of with parameters and . The parameters and are usually assumed in the order of .A factor graph is established for the MPBSBL algorithm in Fig. 2, where , and denote , , , and . The extra variable is introduced and the constraint is represented by . Then, is a function of and , i.e., . The MPBSBL algorithm performed on the factor graph in Fig. 2 is briefed as follows
Denote as the iteration index and as the product of all the incoming messages from to . Then, the variance and mean of are,
(5) 
(6) 
The variance and mean of are updated as
(7) 
The variance and mean from to are
(8) 
Then is updated by the MF message passing,
(9) 
The variance and mean of are
(10) 
Then, is updated by MF message passing,
(11) 
If , user is detected as inactive. Otherwise, is the estimated channel gain.
Iii DNNAided MPBSBL Algorithm


Index  Layer Input  Layer Output  Length 
1  None  
2  
3  
4  
5  
6  
7  
8  
9  

The factor graph in Fig. 2 is densely connected, which results in the correlation problem of the Gaussian messages [13, 14, 15]. To address this problem, we propose a DNNMPBSBL algorithm to imposes weights on the Gaussian message update and the MF message update. To facilitate the training, the message passing is transferred from the factor graph to a DNN in Fig. 3(a). Each iteration of the MPBSBL algorithm is now represented by one iteration block. Within each iteration block, one layer represents one particular message. Two auxiliary layers and are also added for illustration clarity. Therefore, as listed in Table I, there are 9 layers in each iteration block. The layer organization is shown in Fig. 3(b). The weighted message passing is represented by a weighting matrix , whose th entry is nonzero if the th input node is connected to the th output node.
Iiia Weighted Message Passing
Layer 1: Layer 1 is the input within one iteration block.
Layer 2: Layer 2 is the auxiliary layer and the output is derived as follows,
(12) 
(13) 
The fraction and operations are performed elementwise while the operation is the matrix multiplication.
Layer 3: The output of Layer 3 is derived as follows,
(14) 
Layer 4: The output of Layer 4 are derived as follows,
(15) 
Layer 5: The output is derived as follows,
(16) 
Layer 6: The output and of Layer 6 is derived as
(17) 
Layer 7: The output of Layer 7 is
(18) 
Layer 8: The output of Layer 8 is
(19) 
Layer 9: The output of Layer 9 is,
(20) 
IiiB Summary of the Proposed DNNMPBSBL Algorithm
Finally, the proposed DNNMPBSBL algorithm is summarized in Algorithm 1. The mean square error (MSE)
is employed as the loss function for the training period, while the Normalized MSE (NMSE)
is considered for the simulations in the testing period.Iv Simulations
Parameters for the simulations are listed in Table II. We consider a crowded NORA system with lowlatency requirement, i.e., . The NMSE performances of the LSAMPSBL esimator [10], the BOMP estimator (with known active user number) [12] and the GAMMSE estimator (with known user activity) are also considered. The NMSE performance of GAMMSE estimator serves as the lower bound.


Parameter  Symbol  Value 
User number  110  
Subcarrier number  8  
Pilot length  11  
Spreading factor  4  
Activation probability for each user  0.1  
UAD threshold  0.1  
Size of training set  
Size of test set  
Size of minibatch  200  
Epoch number  20  
Learning rate  

The simulation results are shown in Fig. 4, in crowded NORA systems, both the MPBSBL algorithm and the BOMP estimator diverge from the NMSE lower bound as SNR increases, and the LSAMPSBL algorithm fails to work even with 50 iterations. By contrast, the DNNMPBSBL algorithm could closely approach the lower bound within a wide range of SNR. Therefore, the DNNMPBSBL algorithm requires fewer iterations and provides better NMSE performance, indicating its advantages in crowded NORA system with lowlatency requirement.
V Conclusions
A DNNMPBSBL algorithm was proposed in this paper for the joint UAD and CE problem in grantfree NORA systems. The iterative message passing process is transferred from a factor graph to a DNN, while weights are imposed on the messages and trained to improve the UAD and CE accuracy. Simulation results showed that the NMSE performance of the DNNMPBSBL algorithm could approach the lower bound in a feasible number of iterations, indicating its advantages for lowlatency NORA systems.
Acknowledgement
This work was supported by the NSFC under grant 61750110529.
References
 [1] 3GPP, “Service Requirements for MachineType Communications,” TS 22.368 V13.1.0, Dec. 2014.
 [2] Z. Wang and V. W. S. Wong, “Optimal access class barring for stationary machine type communication devices with timing advance information,” IEEE Trans. Wireless Commun., vol.14, no.10, pp.53745387, Oct. 2015.
 [3] K.D. Lee, S. Kim, and B. Yi, “Throughput comparison of random access methods for M2M service over LTE networks,” Proc. 2011 IEEE GLOBECOM Wkshps., Dec. 2011, pp. 373377.
 [4] S. Choi, W. Lee, D. Kim, K. J. Park, S. Choi, and K. Y. Han, “Automatic configuration of random access channel parameters in LTE systems,” Proc. 2011 IFIP Wireless Days, Oct. 2011, pp. 16.
 [5] X. Xu, X. Rao, and V. K. N. Lau, “Active user detection and channel estimation in uplink CRAN systems,”IEEE Inter. Conf. Commun. (ICC), June 2015, pp. 27272732.
 [6] G. Hannak, M. Mayer, A. Jung, G. Matz, and N. Goertz, “Joint channel estimation and activity detection for multiuser communication systems,” IEEE Inter. Conf. Commun. (ICC) Workshop, June 2015, pp. 20862091.
 [7] Z. Chen and W. Yu, “Massive device activity detection by approximate message passing,” in IEEE Inter. Conf. Acoustics, Speech, Signal Processing (ICASSP), Mar. 2017, pp. 35143518.
 [8] L. Liu and W. Yu, “Massive connectivity with massive MIMOpart I: device activity detection and channel estimation,” IEEE Transactions on Signal Processing, vol. 66, no. 11, pp. 29332946.
 [9] M. AlShoukairi and B. Rao, “Sparse Bayesian learning using approximate message passing,” 48th Asilomar Conference on Signals, Systems and Computers, 2014, pp. 19571961.
 [10] C. Wei, H. Liu, Z. Zhang, J. Dang, and L. Wu, “Nearoptimum sparse channel estimation based on least squares and approximate message passing,” IEEE Wireless Communications Letters, vol. 6, no. 6, pp. 754757.
 [11] F. Wei, W. Chen, Y. Wu, J. Ma, and T. A. Tsiftsis, “Messagepassing receiver design for joint channel estimation and data decoding in uplink grantfree SCMA systems,” IEEE Trans. Wireless Commun., vol. 18, no. 1, pp. 167181, Jan. 2019.
 [12] Y. Zhang, Q. Guo, Z. Wang, J. Xi, and N. Wu, “Block sparse Bayesian learning based joint user activity detection and channel estimation for grantfree NOMA systems,” IEEE Transactions on Vehicular Technology, vol. 67, no. 10, pp. 96319640.
 [13] L. Liu, C. Yuen, Y. L. Guan, Y. Li, and Y. Su, “Convergence analysis and assurance for Gaussian message passing iterative detector in massive MUMIMO systems,” IEEE Trans. Wireless Commun., vol. 15, no. 9, pp. 64876500.
 [14] C. Huang, L. Liu, C. Yuen and S. Sun, “Iterative channel estimation using LSE and sparse message passing for MmWave MIMO systems,” IEEE Transactions on Signal Processing, vol. 67, no. 1, pp. 245259, 1 Jan.1, 2019.
 [15] C. Huang, L. Liu, and C. Yuen, “Asymptotically optimal estimation algorithm for the sparse signal with arbitrary distributions,” IEEE Transactions on Vehicular Technology, vol. 67, no. 10, pp. 1007010075, Oct. 2018.