A STUDY OF TRELLIS-CODED QAM ON ISI CHANNELS

Abstract

In recent years, the ever increasing demand for reliable high-speed information transmission has spurred an active interest in the development of efficient coded communication systems, both in terms of power efficiency and bandwidth (spectral) efficiency." The traditional coding schemes trade bandwidth efficiency for increased power efficiency/coding gain and therefore are considered to be suitable only for power limited channels, where bandwidth is abundant. The rapid growth in digital radio communication and high-speed digital networks, demands a large spectral efficiency to meet the enhanced information rate requirements. Ungerboeck's combined coding and modulation scheme, namely Trellis coded modulation (TCM), has provided an impetus to achieve high spectral efficiency on bandlimited channels, while providing adequate power efficiency/coding gain. Over the past decade, the TCM has evolved as a robust and efficient coding scheme for information transmission without sacrificing data rate, bandwidth, and signal power. The fact that it promised to fill most of the 9 dB gap between the rates achievable and Shannon's channel capacity limit, prompted an active research as well as wide-spread practical applications of the TCM. Ungerboeck's TCM scheme is an integrated design approach that regards coding and modulation as a single entity. Essentially, the TCM scheme employs non-binary redundant modulation in conjunction with a finite-state convolutional encoder thai governs the selection of channel signals. The key to this unified design approach is a special mapping technique called mapping by set-partitioning, which ensures that the minimum Euclidean distance (ED) between the coded signal sequences is maximized. At the receiver, the noise corrupted signal sequence is decoded by soft-decision maximum-likelihood Viterbi decoder, which is an optimum receiver. Depending upon the code employed, a TCM scheme can improve the robustness of digital transmission on additive white Gaussian noise (AWGN) channel by 3 to 6 dB relative to an uncoded modulation system, without compromising the bandwidth efficiency or power efficiency. The present work is concerned with the study of Trellis-coded QAM system on AWGN channels as well as on time-dispersive intersymbol interference (ISI) channels. In an effort to overcome the problem of computational complexity of the optimal receiver structure for bandlimited channels, various reduced complexity sub-optimum receiver structures have been considered and their performance evaluated. We consider first, the performance of Trellis-coded QAM systems on AWGN channels, which depends on the decoding algorithm and the distance properties of the code employed. An exact analysis of the error probability is difficult, and one normally resorts to either simulation or the evaluation of performance bounds. Simulation is a time consuming process and therefore cannot provide realistic estimates of error performance at high signal-to-noise ratio (SNR). On the other hand, the performance bounds are the most effective tools in the evaluation of performance at moderate to high SNR. The algorithms used to compute the performance need to be fast and efficient. Most of the algorithms proposed in the literature are based on the transfer function approach, which when combined with union bound yields a tight upper bound to the error probability. In the present work, we have proposed a unidirectional trellis search algorithm based on the shortest-route principle, that computes the distance spectrum of the code using a trellis structure whose state complexity is same as that of the encoder. The algorithm is fast and efficient. The derivation of tight upper bound to error probability requires computation of the complete distance spectrum of the code to evaluate all the terms of the union bounds. In practice, the first few spectral lines of the code are computed to yield a moderately tight upper bound to error probability. Following this approach, the performance of trellis codes on AWGN channel have been evaluated through bounds, using the u proposed distance spectrum computing algorithm, and the results so derived are compared with simulation results. The performance evaluation so derived is found to be nearly tight, in the sense that the simulation result lies well within the computed lower and upper bounds. The proposed method, for the computation of distance spectrum and hence the evaluation of the code performance, is found to be quite effective for Ungerboeck's TCM codes. We observe that, over AWGN channel the TCM schemes achieve a coding gain of nearly 3-5 dB relative to the uncoded reference system. The large spectral efficiency and high coding gain achieved by TCM on AWGN channels, has stimulated researchers to investigate its performance and consider its application to high-speed data transmission over bandlimited time-dispersive channels. The primary impediments over such a channel are the ISI and AWGN. While TCM can effectively enhance noise immunity without a reduction in data rate, a powerful equalization technique such as maximum-likelihood sequence estimation (MLSE) is desired to mitigate the effects of ISI. Therefore for bandlimited channels, a TCM scheme in combination with an optimum MLSE promises to achieve data rates close to channel capacity. The cascade of TCM encoder and ISI channel can be represented by a combined finite-state machine and hence a combined ISICode trellis, whose states is the product of encoder states and ISI stales. Consequently, the receiver performs a maximum-likelihood estimation of data sequence using the Viterbi algorithm (VA) that searches for a minimum cost path in the combined ISI-Code trellis and the resulting structure is optimum. A study has been performed to evaluate the error rate performance of the combined ISI-Code receiver structures, used for the decoding of various Trellis-coded QAM signals in the presence of ISI and AWGN, through simulation. Making use of the error structure of the basic trellis-code employed and the ISI channel characteristics, the performance of the optimum combined MLSE receiver structures have also been evaluated through bounds, and are found to be in concurrence with the simulation results. From the simulation results, we find that the performance of the optimum in combined MLSE receiver structures, over certain ISI channels, is very close to that under an ISI-free environment. Although there is a degradation of the order of 1-2 dB relative to the ISI-free performance, the combined ISI-code trellis structure achieves gain of about 2-3 dB over the uncoded MLSE reference system. Since the computational complexity of the optimum combined MLSE receiver structure grows exponentially with channel memory length the practical implementation becomes prohibitive, even for moderate ISI. This has motivated researchers to find the sub-optimum TCM receiver structures with reduced complexity, that maintains most of the performance advantages of the MLSE. The complexity can be reduced drastically by employing pre-filtering techniques prior to MLSE, as is being used for uncoded modulation schemes. We have proposed, a suboptimum KFE-MLSE receiver structure comprising of Kalman filter equalizer (KFE) followed by maximum-likelihood Viterbi decoder for the decoding of trellis-coded QAM signals in the presence of ISI and AWGN. The proposed KFE-MLSE structure is a sub-optimum receiver structure and the performance degradation for the above can be evaluated by assuming that the Viterbi algorithm still operates with a white Gaussian noise, whose variance is the overall variance of the correlated noise and the residual ISI at the equalizer output. The effect of prefiltering on the free distance of the code can be computed by finding the combined channel-KFE impulse response through 'innovations' representation of the processes involved, and the spectral factorization techniques. We have evaluated the performance of the KFE-MLSE receiver structure, by finding the performance degradation relative to its performance under an ISI-free environment. The performance' bounds so derived are compared with the simulation results of several trellis-coded QAM schemes employing KFE-MLSE structure on different ISI channels. Also, the performance of the KFE-MLSE receiver structure is compared with that of the optimum combined ISIcode trellis structure for limited ISI memory length. The proposed suboptimal KFEMLSE structure achieves significant gain of about 2-2.5 dB over the uncoded KFE iv reference system, although it suffers a performance loss of about 1.0-2.0 dB relative to the optimum combined MLSE structure. An alternative to the prefiltering technique is the use of reduced-slate algorithms, which aim at reducing the states of the optimum combined ISI-code trellis by incorporating a built-in decision-feedback mechanism within the Viterbi decoder. The Reduced State Sequence Estimation (RSSE) and the Parallel Decision Feedback Decoding (PDFD) are two such sequence estimation algorithm, which provide a good performance/complexity trade-off, the latter being a special case of RSSE. In RSSE, the complexity reduction is achieved by using the channel truncation technique and applying the set-partitioning principles inherent in TCM. The slate reduction in RSSE depends on the code, the channel truncation length, and the depth of set-partitioning of the signal constellation. A family of reduced slate sequence estimators can be obtained for a given TCM code and a given ISI channel, offering a wide trade-off between decoding complexity and performance. The complexity of RSSE trellis can range between that of the encoder trellis and the combined MLSE trellis. When the complexity of the RSSE trellis is reduced to thai of the TCM encoder trellis, the receiver structure is referred to as the parallel decision feedback decoding (PDFD). We have considered the study of several RSSE and PDFD receiver structures for the decoding of trellis-coded QAM with different orders of complexity reduction, for various ISI channels. The performance of these structures have been studied by simulation and are compared with the error performance of other receiver structures, considered earlier. The RSSE receiver structures exhibit improved performance over KFE-MLSE receiver structure, but at the cost of increased complexity. For channel equalization, the optimum combined MLSE receiver structure (implemented by VA) or its sub-optimum variants, require an exact knowledge of the channel characteristics, especially when the channel parameters are time-varying. A wide range of adaptive algorithms for channel estimation have been reported in the literature, the most common being the LMS algorithm and the RLS algorithm. The decision delay inherent in the VA causes poor tracking performance, particularly when the channel characteristics are rapidly time-varying. To circumvent Ihis problem, a new channel estimation procedure has been reported in recent years, where the adaptive channel estimation is accomplished for each state in the VA, using the zero-delay decisions associated with its survivor path to update the channel coefficients. We next consider the adaptive implementation of the receiver structures discussed earlier, for the decoding of trellis-coded QAM on time-variant ISI channels. We have implemented various adaptive receiver structures employing LMS and RLS channel estimators, using both delayed-decision updating and delay-tree updating of channel coefficients. The tracking characteristics of the adaptive algorithms have been studied for various random time-variant ISI channels. The mean-square error performance of the adaptive channel estimator have been studied by simulation. Also, the error performance of the various adaptive receiver structures have been evaluated through simulation. In recent years, there has been an increasing interest in high-speed digital transmission over cellular mobile radio, and mobile satellite channels, which are characterized as multipath fading channels with time-dispersion. One of the most efficient technique to reduce the effect of fading is through the use of diversity reception, where the receiver is provided with several replicas of the same information transmitted over D-independently fading channels. We have next considered a study on the performance of different trellis-coded QAM schemes over fading dispersive channels. The error performance of the adaptive RSSE, PDFD and KFE-MLSE receiver structures have been studietl by simulation for different fade rates. Using D-diversity reception to combat severe fading, the error performance of the above adaptive receiver structures have been determined for different orders of diversity D.