Jar Decoding: Non-Asymptotic Converse Coding Theorems, Taylor-Type Expansion, and Optimality

Recently, a new decoding rule called jar decoding was proposed, under which the decoder first forms a set of suitable size, called a jar, consisting of sequences from the channel input alphabet considered to be closely related to the received channel output sequence through the channel, and then takes any codeword from the jar as the estimate of the transmitted codeword; under jar decoding, a nonasymptotic achievable tradeoff between the coding rate and word error probability was also established for any discrete input memoryless channel with discrete or continuous output (DIMC). Along the path of non-asymptotic analysis, in this paper, it is further shown that jar decoding is actually optimal up to the second order coding performance by establishing new non-asymptotic converse coding theorems, and determining the (best) coding performance of finite block length for any block length n and word error probability up to the second order. Specifically, a new converse proof technique dubbed the outer mirror image of jar is first presented and used to establish new non-asymptotic converse coding theorems for any encoding and decoding scheme. To determine the coding performance of finite block length for any block length n and error probability , a quantity δt,n( ) is then defined to measure the relative magnitude of the error probability and block length n with respect to a given channel and an input distribution t. By combining the achievability of jar decoding and the new converses, it is demonstrated that when < 1/2, the best channel coding rate Rn( ) given n and has a “Taylor-type expansion” with respect to δt,n( ), where the first two terms of the expansion are maxt[I(t;P )−δt,n( )], which is equal to I(t∗, P ) − δt∗,n( ) for some optimal distribution t∗, and the third order term of the expansion is O(δ t∗,n( )) whenever δt∗,n( ) = Ω( √ lnn/n), thus implying the optimality of jar decoding up to the second order coding performance. Finally, based on the Taylor-type expansion and the new converses, two approximation formulas for Rn( ) (dubbed “SO” and “NEP”) are provided; they are further evaluated and compared against some of the best bounds known so far, as well as the normal approximation of Rn( ) revisited recently in the literature. It turns out that while the normal approximation is all over the map, i.e. sometime below achievable bounds and sometime above converse bounds, the SO approximation is much more reliable as it is always below converses; in the meantime, the NEP approximation is the best among the three and always provides an accurate estimation for Rn( ). An important implication arising from the Taylor-type expansion of Rn( ) is that in the practical non-asymptotic regime, the optimal marginal codeword symbol distribution is not necessarily a capacity achieving distribution.