Paper detail

Variable Selection is Hard

Variable selection for sparse linear regression is the problem of finding, given an m x p matrix B and a target vector y, a sparse vector x such that Bx approximately equals y. Assuming a standard complexity hypothesis, we show that no polynomial-time algorithm can find a k&#39;-sparse x with ||Bx-y||^2<=h(m,p), where k&#39;=k*2^{log^{1-delta} p} and h(m,p)<=p^(C_1)*m^(1-C_2), where delta>0, C_1>0,C_2>0 are arbitrary. This is true even under the promise that there is an unknown k-sparse vector x^* satisfying Bx^*=y. We prove a similar result for a statistical version of the problem in which the data are corrupted by noise. To the authors&#39; knowledge, these are the first hardness results for sparse regression that apply when the algorithm simultaneously has k&#39;>k and h(m,p)>0.