Channels, Codes and Capacity

In this chapter we introduce our task: communicating a digital message without error (or with as few errors as possible) despite an imperfect communications medium. Figure 1.1 shows a typical communications system. In this text we will assume that our source is producing binary data, but it could equally be an analog source followed by analog-to-digital conversion. Through the early 1940s, engineers designing the first digital communications systems, based on pulse code modulation, worked on the assumption that information could be transmitted usefully in digital form over noise-corrupted communication channels but only in such a way that the transmission was unavoidably compromised. The effects of noise could be managed, it was believed, only by increasing the transmitted signal power enough to ensure that the received signal-to-noise ratio was sufficiently high. Shannon’s revolutionary 1948 work changed this view in a fundamental way, showing that it is possible to transmit digital data with arbitrarily high reliability, over noise-corrupted channels, by encoding the digital message with an error correction code prior to transmission and subsequently decoding it at the receiver. The error correction encoder maps each vector of K digits representing the message to longer vectors of N digits known as codewords. The redundancy implicit in the transmission of codewords, rather than the raw data alone, is the quid pro quo for achieving reliable communication over intrinsically unreliable channels. The code rate r = K/N defines the amount of redundancy added by the error correction code. The transmitted bits may be corrupted in some way by the channel, and it is the function of the error correction decoder to use the added redundancy to determine the corresponding K message bits despite the imperfect reception. In this chapter we will introduce the basic ideas behind error correction. In Section 1.1 we describe the channels considered in this text and in Section 1.2 the fundamental limits to communicating on those channels. Finally, in Section 1.3 we introduce error correction techniques.