On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems

We investigate the computational complexity of two closely related classes of combinatorial optimization problems for linear systems which arise in various elds such as machine learning, operations research and pattern recognition. In the rst class (Min ULR) one wishes, given a possibly infeasible system of linear relations, to nd a solution that violates as few relations as possible while satisfying all the others. In the second class (Min RVLS) the linear system is supposed to be feasible and one looks for a solution with as few nonzero variables as possible. For both Min ULR and Min RVLS the four basic types of relational operators =, , > and 6 = are considered. While Min RVLS with equations was known to be NP-hard in 27], we established in 2, 5] that Min ULR with equalities and inequalities are NP-hard even when restricted to homogeneous systems with bipolar coeecients. The latter problems have been shown hard to approximate in 7]. In this paper we determine strong bounds on the approximability of various variants of Min RVLS and Min ULR, including constrained ones where the variables are restricted to take binary values or where some relations are mandatory while others are optional. The various NP-hard versions turn out to have diierent approximability properties depending on the type of relations and the additional constraints, but none of them can be approximated within any constant factor, unless P=NP. Particular attention is devoted to two interesting special cases that occur in discriminant analysis and machine learning. In particular, we disprove a conjecture in 64] regarding the existence of a polynomial time algorithm to design linear classiiers (or perceptrons) that involve a close-to-minimum number of features.