On the Computational Complexity of 2D Maximum-Likelihood Sequence Detection

We consider a two-dimensional version of the classical maximum-likelihood sequence detection problem for a binary antipodal signal that is corrupted by linear intersymbol interference (ISI) and then passed through a memoryless channel. For one-dimensional signals and fixed ISI, this detection problem is well-known to be efficiently solved using the Viterbi algorithm. We show that the two-dimensional version is, in general, intractable, in the sense that a decision formulation of the problem is NP complete for a certain fixed two-dimensional ISI and memoryless channel with errors and erasures. We also extend the result to the additive white Gaussian noise channel and the same fixed ISI as in the previous case. Finally, we show how our proof of NP completeness for the two-dimensional case can also be used to prove the NP completeness of multiuser detection under a Toeplitz constraint, shown in [4] to be equivalent to a variant of one-dimensional maximum likelihood sequence detection when the ISI is not fixed, thereby proving Conjecture 1 in [4].