Source author record

Hongbo Dong

Hongbo Dong appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

math.OC Machine Learning math.NA Methodology

Catalog footprint

What is connected

4works

4topics

3close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2016arXiv

On the exact recovery of sparse signals via conic relaxations

In this note we compare two recently proposed semidefinite relaxations for the sparse linear regression problem by Pilanci, Wainwright and El Ghaoui (Sparse learning via boolean relaxations, 2015) and Dong, Chen and Linderoth (Relaxation vs. Regularization A conic optimization perspective of statistical variable selection, 2015). We focus on the cardinality constrained formulation, and prove that the relaxation proposed by Dong, etc. is theoretically no weaker than the one proposed by Pilanci, etc. Therefore any sufficient condition of exact recovery derived by Pilanci can be readily applied to the other relaxation, including their results on high probability recovery for Gaussian ensemble. Finally we provide empirical evidence that the relaxation by Dong, etc. requires much fewer observations to guarantee the recovery of true support.

preprint2015arXiv

Regularization vs. Relaxation: A conic optimization perspective of statistical variable selection

Variable selection is a fundamental task in statistical data analysis. Sparsity-inducing regularization methods are a popular class of methods that simultaneously perform variable selection and model estimation. The central problem is a quadratic optimization problem with an l0-norm penalty. Exactly enforcing the l0-norm penalty is computationally intractable for larger scale problems, so dif- ferent sparsity-inducing penalty functions that approximate the l0-norm have been introduced. In this paper, we show that viewing the problem from a convex relaxation perspective offers new insights. In particular, we show that a popular sparsity-inducing concave penalty function known as the Minimax Concave Penalty (MCP), and the reverse Huber penalty derived in a recent work by Pilanci, Wainwright and Ghaoui, can both be derived as special cases of a lifted convex relaxation called the perspective relaxation. The optimal perspective relaxation is a related minimax problem that balances the overall convexity and tightness of approximation to the l0 norm. We show it can be solved by a semidefinite relaxation. Moreover, a probabilistic interpretation of the semidefinite relaxation reveals connections with the boolean quadric polytope in combinatorial optimization. Finally by reformulating the l0-norm pe- nalized problem as a two-level problem, with the inner level being a Max-Cut problem, our proposed semidefinite relaxation can be realized by replacing the inner level problem with its semidefinite relaxation studied by Goemans and Williamson. This interpretation suggests using the Goemans-Williamson rounding procedure to find approximate solutions to the l0-norm penalized problem. Numerical experiments demonstrate the tightness of our proposed semidefinite relaxation, and the effectiveness of finding approximate solutions by Goemans-Williamson rounding.

preprint2014arXiv

Relaxing nonconvex quadratic functions by multiple adaptive diagonal perturbations

The current bottleneck of globally solving mixed-integer (non-convex) quadratically constrained problem (MIQCP) is still to construct strong but computationally cheap convex relaxations, especially when dense quadratic functions are present. We propose a cutting surface procedure based on multiple diagonal perturbations to derive strong convex quadratic relaxations for nonconvex quadratic problem with separable constraints. Our resulting relaxation does not use significantly more variables than the original problem, in contrast to many other relaxations based on lifting. The corresponding separation problem is a highly structured semidefinite program (SDP) with convex but non-smooth objective. We propose to solve this separation problem with a specialized primal-barrier coordinate minimization algorithm. Computational results show that our approach is very promising. First, our separation algorithm is at least an order of magnitude faster than interior point methods for SDPs on problems up to a few hundred variables. Secondly, on nonconvex quadratic integer problems, our cutting surface procedure provides lower bounds of almost the same strength with the diagonal SDP bounds used by (Buchheim and Wiegele, 2013) in their branch-and-bound code Q-MIST, while our procedure is at least an order of magnitude faster on problems with dimension greater than 70. Finally, combined with (linear) projected RLT cutting planes proposed by (Saxena, Bonami and Lee, 2011), our procedure provides slightly weaker bounds than their projected SDP+RLT cutting surface procedure, but in several order of magnitude shorter time. Finally we discuss various avenues to extend our work to design more efficient branch-and-bound algorithms for MIQCPs.

preprint2012arXiv

Reduced rank regression via adaptive nuclear norm penalization

Adaptive nuclear-norm penalization is proposed for low-rank matrix approximation, by which we develop a new reduced-rank estimation method for the general high-dimensional multivariate regression problems. The adaptive nuclear norm of a matrix is defined as the weighted sum of the singular values of the matrix. For example, the pre-specified weights may be some negative power of the singular values of the data matrix (or its projection in regression setting). The adaptive nuclear norm is generally non-convex under the natural restriction that the weight decreases with the singular value. However, we show that the proposed non-convex penalized regression method has a global optimal solution obtained from an adaptively soft-thresholded singular value decomposition. This new reduced-rank estimator is computationally efficient, has continuous solution path and possesses better bias-variance property than its classical counterpart. The rank consistency and prediction/estimation performance bounds of the proposed estimator are established under high-dimensional asymptotic regime. Simulation studies and an application in genetics demonstrate that the proposed estimator has superior performance to several existing methods. The adaptive nuclear-norm penalization can also serve as a building block to study a broad class of singular value penalties.

Institution

Affiliation not imported yet

This author record came from a source that does not expose affiliation metadata. Once the author claims the profile or we enrich the record from another provider, this section will link to the concrete institution.

Topic footprint