AI‐Enhanced Signal Detection and Channel Estimation for Beyond 5G and 6G Wireless Networks

Integrating deep learning (DL) with massive multiple input multiple output (ma-MIMO) technology has provided a framework for designing new communication systems for next-generation technologies such as sixth-generation (6G) networks. However, due to huge transmitting and receiving antenna sizes, channel estimation and signal detection become a very challenging job at the receiver side. To address the channel estimation and signal detection problem in ma-MIMO systems, this paper presents two system frameworks by considering the two scenarios based on channel information at the receiver end. In scenario 1, the Channel matrix (Formula presented.) is unknown at the base station (BS), and to ensure accurate channel estimation, the pilot symbols are integrated with the transmitted symbol in the ma-MIMO systems. Based on Scenario 1, this paper proposes an optimized pilot-assisted feedforward network for channel estimation called FF-PCNet in the ma-MIMO system. In scenario 2, the channel matrix (Formula presented.) is fully known at the BS and uses this exact information for signal detection in the ma-MIMO systems. Based on scenario 2, this paper proposes two methods for signal detection in ma-MIMO systems. The proposed method-1 is based on an optimized long short-term memory-based detection network called LSTM-DetNet, and method-2 is based on an optimized and customized feed-forward detection network called FF-DetNet for signal detection in the ma-MIMO systems. Numerical results show that, for channel estimation, the FF-PCNet performs excellently and achieves a 40.2% low average error per symbol compared to the benchmark traditional MIMO estimator known as least squares estimation (LSE). For signal detection, although method 1, known as LSTM-DetNet, achieves better performance than other benchmark MIMO detectors, yet unable to beat the AIDETECT MIMO detector. However, our second proposed method, known as FF-DetNet, not only achieves better SER performance ranging between 73.2% to 99.993% for both MIMO and ma-MIMO systems but has also been able to achieve much lower computational complexity compared to benchmark artificial intelligence (AI)-based MIMO detectors.