Providing efficient support for the Internet of Things (IoT) is one of the major objectives for the cellular wireless communication. Machine-to-Machine (M2M) communication is anticipated to support billions of Machine Type Communication (MTC) devices. In addition, the MTC devices are sporadically activated with short packets . Therefore, the random access (RA) process for M2M communications in IoT is characterized by massive connection and sporadic transmission, as well as small-sized data packets.
Confronted with the characteristics described above, the conventional orthogonal multiple access (OMA) scheme becomes infeasible due to its low resource efficiency. To facilitate the sharing of uplink resources, different RA schemes were proposed and can be generally categorized into two types: grant-based RA [2, 3, 4] and grant-free RA [5, 6, 7, 8, 9, 10, 11, 12].
I-a Grant-Based Random Access
In grant-based RA schemes, activated users contend for the RBs by transmitting a preamble sequence to the base station (BS), while a RB is assigned to the activated user, whose preamble sequence is received and accepted by the BS. One problem with the grant-based RA schemes is that the RB is wasted when more than one active device transmit the same preamble sequence. Some solutions were proposed to alleviate the RA congestion by reducing the collision probability, such as the Access Class Barring (ACB) scheme, delicate splitting of the RA preamble set , and automatic configuration of the RA parameters . However, the RB wastage cannot be fully avoided by grant-based RA schemes, which results in low resource efficiency. Furthermore, a handshaking process is always to recognize the contention winner, which undermines the uplink transmission efficiency of small data packets.
I-B Grant-Free Random Access
I-B1 Compressed sensing-based grant-free RA schemes
Compressed sensing (CS) algorithms employ pilot sequences to accomplish the user activity detection (UAD) and/or channel estimation (CE) problem. For example, the joint UAD and CE problem was addressed by a modified Bayesian compressed sensing algorithm  for the cloud radio access network (C-RAN). In addition, the powerful approximate message passing (AMP) algorithm was employed for the joint UAD and CE problem when the BS is equipped either with a single antenna [6, 7] or with multiple antennas .
I-B2 Sparse Bayesian learning-based grant-free RA schemes
The sparse Bayesian learning (SBL) algorithm considers the prior hyper-parameter of the sparse signal. The Expectation Maximization (EM) method was employed by the AMP-SBL algorithm to update the sparse signal and the hyper-parameter. A least square (LS)-based AMP-SBL (LS-AMP-SBL) algorithm  was proposed to recover the sparse signal in three steps. Recently, a message-passing receiver design was proposed for the joint channel estimation and data decoding in uplink grant-free SCMA systems . In addition, a message passing-based block sparse Bayesian learning (MP-BSBL) algorithm  was proposed for a grant-free NOMA system.
In this paper, we consider a LDS-OFDM system, where devices perform grant-free RA once they are activated. A deep neural network-aided message passing-based block sparse Bayesian learning (DNN-MP-BSBL) algorithm is proposed to perform joint UAD and CE. The iterative message passing process of the MP-BSBL algorithm  is transferred from a factor graph to a neural network. Weights are imposed on the messages passing in the neural network and trained to minimize the estimation error.
The rest of this paper is organized as follows. The system model and the MP-BSBL algorithm are presented in Section II. The DNN structure for the DNN-MP-BSBL algorithm is illustrated in Section III, where the weighted message passing is explained in details. Simulation results are given in Section IV to verify the UAD and CE accuracy of the proposed DNN-MP-BSBL algorithm. Finally, Section V concludes this paper.
Ii Joint UAD and CE by MP-BSBL
Ii-a System Model and Problem Formulation
A LDS-OFDM system is considered in Fig. 1. There are sub-carriers and users. Each user is activated with probability . For active user , its information sequence is encoded and mapped into a QAM sequence with length . ZC sequences are adopted as pilot sequences. One unique sequence with length is allocated for user , and inserted into the transmitted sequence , i.e., . Therefore, the length of is . The LDS spreader for user is characterized by
, which is a sparse vector with lengthand non-zero elements. The LDS spreaders for all the users are characterized by a LDS spreading matrix . We consider a regular , i.e., the column degree and the row degree in are constant. Each sub-carrier is shared by potential users, with . When multiple users are active on the same sub-carrier, the RA is conducted in a NOMA manner. After the OFDM de-modulator, the received matrix is
where the -th entry of represents the -th received symbol on the -th sub-carrier, the -th entry of represents the -th transmitted symbol of the -th user, and is a row sparse channel matrix, which integrates the activity of the users. The activity indicator if user is inactive. Otherwise, and the -th row of represents the channel gain vector on sub-carriers for user . The entries in the AWGN matrix
are assumed i.i.d with noise variance. can be decomposed as , where and represent the received matrices w.r.t. the pilot sequences and the data sequences, respectively. We consider to solve the joint UAD and CE problem,
where is assumed known to the receiver. Then we perform vectorization on the transpose of as in ,
where , , and is the channel gain vector of user on sub-carriers. According to the LDS spreading matrix , the transmitted symbols of each user are only spread onto sub-carriers. Therefore, elements in are zero. We further simplify by eliminating the zeros. Accordingly, the columns in corresponding to the zeros in are also removed. Finally, we obtain the simplified version of (2) in equation () of (3). According to (3), the joint UAD and CE problem is equivalent to recovering .
Ii-B MP-BSBL Algorithm 
The recovery of can be addressed by the MP-BSBL algorithm .
For user , the distribution of is assumed conditioned on a hyper-parameter , i.e., . The hyper-parameter
is assumed to follow a Gamma distribution. The noise precisionis unknown at the receiver but assumed with a priori probability
. With these assumptions above, the joint a posterior probability is factorized as follows
where , , is the -th row of matrix in (3), , , and ,with mean and variance , while represents the Gamma distribution pdf of with parameters and . The parameters and are usually assumed in the order of .
A factor graph is established for the MP-BSBL algorithm in Fig. 2, where , and denote , , , and . The extra variable is introduced and the constraint is represented by . Then, is a function of and , i.e., . The MP-BSBL algorithm performed on the factor graph in Fig. 2 is briefed as follows
Denote as the iteration index and as the product of all the incoming messages from to . Then, the variance and mean of are,
The variance and mean of are updated as
The variance and mean from to are
Then is updated by the MF message passing,
The variance and mean of are
Then, is updated by MF message passing,
If , user is detected as inactive. Otherwise, is the estimated channel gain.
Iii DNN-Aided MP-BSBL Algorithm
|Index||Layer Input||Layer Output||Length|
The factor graph in Fig. 2 is densely connected, which results in the correlation problem of the Gaussian messages [13, 14, 15]. To address this problem, we propose a DNN-MP-BSBL algorithm to imposes weights on the Gaussian message update and the MF message update. To facilitate the training, the message passing is transferred from the factor graph to a DNN in Fig. 3(a). Each iteration of the MP-BSBL algorithm is now represented by one iteration block. Within each iteration block, one layer represents one particular message. Two auxiliary layers and are also added for illustration clarity. Therefore, as listed in Table I, there are 9 layers in each iteration block. The layer organization is shown in Fig. 3(b). The weighted message passing is represented by a weighting matrix , whose -th entry is non-zero if the -th input node is connected to the -th output node.
Iii-a Weighted Message Passing
Layer 1: Layer 1 is the input within one iteration block.
Layer 2: Layer 2 is the auxiliary layer and the output is derived as follows,
The fraction and operations are performed elementwise while the operation is the matrix multiplication.
Layer 3: The output of Layer 3 is derived as follows,
Layer 4: The output of Layer 4 are derived as follows,
Layer 5: The output is derived as follows,
Layer 6: The output and of Layer 6 is derived as
Layer 7: The output of Layer 7 is
Layer 8: The output of Layer 8 is
Layer 9: The output of Layer 9 is,
Iii-B Summary of the Proposed DNN-MP-BSBL Algorithm
Parameters for the simulations are listed in Table II. We consider a crowded NORA system with low-latency requirement, i.e., . The NMSE performances of the LS-AMP-SBL esimator , the BOMP estimator (with known active user number)  and the GA-MMSE estimator (with known user activity) are also considered. The NMSE performance of GA-MMSE estimator serves as the lower bound.
|Activation probability for each user||0.1|
|Size of training set|
|Size of test set|
|Size of mini-batch||200|
The simulation results are shown in Fig. 4, in crowded NORA systems, both the MP-BSBL algorithm and the BOMP estimator diverge from the NMSE lower bound as SNR increases, and the LS-AMP-SBL algorithm fails to work even with 50 iterations. By contrast, the DNN-MP-BSBL algorithm could closely approach the lower bound within a wide range of SNR. Therefore, the DNN-MP-BSBL algorithm requires fewer iterations and provides better NMSE performance, indicating its advantages in crowded NORA system with low-latency requirement.
A DNN-MP-BSBL algorithm was proposed in this paper for the joint UAD and CE problem in grant-free NORA systems. The iterative message passing process is transferred from a factor graph to a DNN, while weights are imposed on the messages and trained to improve the UAD and CE accuracy. Simulation results showed that the NMSE performance of the DNN-MP-BSBL algorithm could approach the lower bound in a feasible number of iterations, indicating its advantages for low-latency NORA systems.
This work was supported by the NSFC under grant 61750110529.
-  3GPP, “Service Requirements for Machine-Type Communications,” TS 22.368 V13.1.0, Dec. 2014.
-  Z. Wang and V. W. S. Wong, “Optimal access class barring for stationary machine type communication devices with timing advance information,” IEEE Trans. Wireless Commun., vol.14, no.10, pp.5374-5387, Oct. 2015.
-  K.-D. Lee, S. Kim, and B. Yi, “Throughput comparison of random access methods for M2M service over LTE networks,” Proc. 2011 IEEE GLOBECOM Wkshps., Dec. 2011, pp. 373-377.
-  S. Choi, W. Lee, D. Kim, K. J. Park, S. Choi, and K. Y. Han, “Automatic configuration of random access channel parameters in LTE systems,” Proc. 2011 IFIP Wireless Days, Oct. 2011, pp. 1-6.
-  X. Xu, X. Rao, and V. K. N. Lau, “Active user detection and channel estimation in uplink CRAN systems,”IEEE Inter. Conf. Commun. (ICC), June 2015, pp. 2727-2732.
-  G. Hannak, M. Mayer, A. Jung, G. Matz, and N. Goertz, “Joint channel estimation and activity detection for multiuser communication systems,” IEEE Inter. Conf. Commun. (ICC) Workshop, June 2015, pp. 2086-2091.
-  Z. Chen and W. Yu, “Massive device activity detection by approximate message passing,” in IEEE Inter. Conf. Acoustics, Speech, Signal Processing (ICASSP), Mar. 2017, pp. 3514-3518.
-  L. Liu and W. Yu, “Massive connectivity with massive MIMO-part I: device activity detection and channel estimation,” IEEE Transactions on Signal Processing, vol. 66, no. 11, pp. 2933-2946.
-  M. Al-Shoukairi and B. Rao, “Sparse Bayesian learning using approximate message passing,” 48th Asilomar Conference on Signals, Systems and Computers, 2014, pp. 1957-1961.
-  C. Wei, H. Liu, Z. Zhang, J. Dang, and L. Wu, “Near-optimum sparse channel estimation based on least squares and approximate message passing,” IEEE Wireless Communications Letters, vol. 6, no. 6, pp. 754-757.
-  F. Wei, W. Chen, Y. Wu, J. Ma, and T. A. Tsiftsis, “Message-passing receiver design for joint channel estimation and data decoding in uplink grant-free SCMA systems,” IEEE Trans. Wireless Commun., vol. 18, no. 1, pp. 167-181, Jan. 2019.
-  Y. Zhang, Q. Guo, Z. Wang, J. Xi, and N. Wu, “Block sparse Bayesian learning based joint user activity detection and channel estimation for grant-free NOMA systems,” IEEE Transactions on Vehicular Technology, vol. 67, no. 10, pp. 9631-9640.
-  L. Liu, C. Yuen, Y. L. Guan, Y. Li, and Y. Su, “Convergence analysis and assurance for Gaussian message passing iterative detector in massive MU-MIMO systems,” IEEE Trans. Wireless Commun., vol. 15, no. 9, pp. 6487-6500.
-  C. Huang, L. Liu, C. Yuen and S. Sun, “Iterative channel estimation using LSE and sparse message passing for MmWave MIMO systems,” IEEE Transactions on Signal Processing, vol. 67, no. 1, pp. 245-259, 1 Jan.1, 2019.
-  C. Huang, L. Liu, and C. Yuen, “Asymptotically optimal estimation algorithm for the sparse signal with arbitrary distributions,” IEEE Transactions on Vehicular Technology, vol. 67, no. 10, pp. 10070-10075, Oct. 2018.