BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Qiyang Han

Qiyang Han appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
math.STStatistics Theorymath.PRInformation Theorymath.ITMachine LearningMethodology

Catalog footprint

What is connected

10works
7topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

10 published item(s)

preprint2022arXiv

Noisy linear inverse problems under convex constraints: Exact risk asymptotics in high dimensions

In the standard Gaussian linear measurement model $Y=Xμ_0+ξ\in \mathbb{R}^m$ with a fixed noise level $σ>0$, we consider the problem of estimating the unknown signal $μ_0$ under a convex constraint $μ_0 \in K$, where $K$ is a closed convex set in $\mathbb{R}^n$. We show that the risk of the natural convex constrained least squares estimator (LSE) $\hatμ(σ)$ can be characterized exactly in high dimensional limits, by that of the convex constrained LSE $\hatμ_K^{\mathsf{seq}}$ in the corresponding Gaussian sequence model at a different noise level. The characterization holds (uniformly) for risks in the maximal regime that ranges from constant order all the way down to essentially the parametric rate, as long as certain necessary non-degeneracy condition is satisfied for $\hatμ(σ)$. The precise risk characterization reveals a fundamental difference between noiseless (or low noise limit) and noisy linear inverse problems in terms of the sample complexity for signal recovery. A concrete example is given by the isotonic regression problem: While exact recovery of a general monotone signal requires $m\gg n^{1/3}$ samples in the noiseless setting, consistent signal recovery in the noisy setting requires as few as $m\gg \log n$ samples. Such a discrepancy occurs when the low and high noise risk behavior of $\hatμ_K^{\mathsf{seq}}$ differ significantly. In statistical languages, this occurs when $\hatμ_K^{\mathsf{seq}}$ estimates $0$ at a faster `adaptation rate' than the slower `worst-case rate' for general signals. Several other examples, including non-negative least squares and generalized Lasso (in constrained forms), are also worked out to demonstrate the concrete applicability of the theory in problems of different types.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Universality of regularized regression estimators in high dimensions

The Convex Gaussian Min-Max Theorem (CGMT) has emerged as a prominent theoretical tool for analyzing the precise stochastic behavior of various statistical estimators in the so-called high dimensional proportional regime, where the sample size and the signal dimension are of the same order. However, a well recognized limitation of the existing CGMT machinery rests in its stringent requirement on the exact Gaussianity of the design matrix, therefore rendering the obtained precise high dimensional asymptotics largely a specific Gaussian theory in various important statistical models. This paper provides a structural universality framework for a broad class of regularized regression estimators that is particularly compatible with the CGMT machinery. In particular, we show that with a good enough $\ell_\infty$ bound for the regression estimator $\hatμ_A$, any `structural property' that can be detected via the CGMT for $\hatμ_G$ (under a standard Gaussian design $G$) also holds for $\hatμ_A$ under a general design $A$ with independent entries. As a proof of concept, we demonstrate our new universality framework in three key examples of regularized regression estimators: the Ridge, Lasso and regularized robust regression estimators, where new universality properties of risk asymptotics and/or distributions of regression estimators and other related quantities are proved. As a major statistical implication of the Lasso universality results, we validate inference procedures using the degrees-of-freedom adjusted debiased Lasso under general design and error distributions. We also provide a counterexample, showing that universality properties for regularized regression estimators do not extend to general isotropic designs.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Multiplier U-processes: sharp bounds and applications

The theory for multiplier empirical processes has been one of the central topics in the development of the classical theory of empirical processes, due to its wide applicability to various statistical problems. In this paper, we develop theory and tools for studying multiplier $U$-processes, a natural higher-order generalization of the multiplier empirical processes. To this end, we develop a multiplier inequality that quantifies the moduli of continuity of the multiplier $U$-process in terms of that of the (decoupled) symmetrized $U$-process. The new inequality finds a variety of applications including (i) multiplier and bootstrap central limit theorems for $U$-processes, (ii) general theory for bootstrap $M$-estimators based on $U$-statistics, and (iii) theory for $M$-estimation under general complex sampling designs, again based on $U$-statistics.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Oracle posterior contraction rates under hierarchical priors

We offer a general Bayes theoretic framework to derive posterior contraction rates under a hierarchical prior design: the first-step prior serves to assess the model selection uncertainty, and the second-step prior quantifies the prior belief on the strength of the signals within the model chosen from the first step. In particular, we establish non-asymptotic oracle posterior contraction rates under (i) a local Gaussianity condition on the log likelihood ratio of the statistical experiment, (ii) a local entropy condition on the dimensionality of the models, and (iii) a sufficient mass condition on the second-step prior near the best approximating signal for each model. The first-step prior can be designed generically. The posterior distribution enjoys Gaussian tail behavior and therefore the resulting posterior mean also satisfies an oracle inequality, automatically serving as an adaptive point estimator in a frequentist sense. Model mis-specification is allowed in these oracle rates. The local Gaussianity condition serves as a unified attempt of non-asymptotic Gaussian quantification of the experiments, and can be easily verified in various experiments considered in [GvdV07a] and beyond. The general results are applied in various problems including: (i) trace regression, (ii) shape-restricted isotonic/convex regression, (iii) high-dimensional partially linear regression, (iv) covariance matrix estimation in the sparse factor model, (v) detection of non-smooth polytopal image boundary, and (vi) intensity estimation in a Poisson point process model. These new results serve either as theoretical justification of practical prior proposals in the literature, or as an illustration of the generic construction scheme of a (nearly) minimax adaptive estimator for a complicated experiment.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Set structured global empirical risk minimizers are rate optimal in general dimensions

Entropy integrals are widely used as a powerful empirical process tool to obtain upper bounds for the rates of convergence of global empirical risk minimizers (ERMs), in standard settings such as density estimation and regression. The upper bound for the convergence rates thus obtained typically matches the minimax lower bound when the entropy integral converges, but admits a strict gap compared to the lower bound when it diverges. Birgé and Massart [BM93] provided a striking example showing that such a gap is real with the entropy structure alone: for a variant of the natural Hölder class with low regularity, the global ERM actually converges at the rate predicted by the entropy integral that substantially deviates from the lower bound. The counter-example has spawned a long-standing negative position on the use of global ERMs in the regime where the entropy integral diverges, as they are heuristically believed to converge at a sub-optimal rate in a variety of models. The present paper demonstrates that this gap can be closed if the models admit certain degree of `set structures' in addition to the entropy structure. In other words, the global ERMs in such set structured models will indeed be rate-optimal, matching the lower bound even when the entropy integral diverges. The models with set structures we investigate include (i) image and edge estimation, (ii) binary classification, (iii) multiple isotonic regression, (iv) $s$-concave density estimation, all in general dimensions when the entropy integral diverges. Here set structures are interpreted broadly in the sense that the complexity of the underlying models can be essentially captured by the size of the empirical process over certain class of measurable sets, for which matching upper and lower bounds are obtained to facilitate the derivation of sharp convergence rates for the associated global ERMs.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Inference for local parameters in convexity constrained models

We consider the problem of inference for local parameters of a convex regression function $f_0: [0,1] \to \mathbb{R}$ based on observations from a standard nonparametric regression model, using the convex least squares estimator (LSE) $\widehat{f}_n$. For $x_0 \in (0,1)$, the local parameters include the pointwise function value $f_0(x_0)$, the pointwise derivative $f_0'(x_0)$, and the anti-mode (i.e., the smallest minimizer) of $f_0$. The existing limiting distribution of the estimation error $(\widehat{f}_n(x_0) - f_0(x_0), \widehat{f}_n'(x_0) - f_0'(x_0) )$ depends on the unknown second derivative $f_0''(x_0)$, and is therefore not directly applicable for inference. To circumvent this impasse, we show that the following locally normalized errors (LNEs) enjoy pivotal limiting behavior: Let $[\widehat{u}(x_0), \widehat{v}(x_0)]$ be the maximal interval containing $x_0$ where $\widehat{f}_n$ is linear. Then, under standard conditions, $$\binom{ \sqrt{n(\widehat{v}(x_0)-\widehat{u}(x_0))}(\widehat{f}_n(x_0)-f_0(x_0)) }{ \sqrt{n(\widehat{v}(x_0)-\widehat{u}(x_0))^3}(\widehat{f}_n'(x_0)-f_0'(x_0))} \rightsquigarrow σ\cdot \binom{\mathbb{L}^{(0)}_2}{\mathbb{L}^{(1)}_2},$$ where $n$ is the sample size, $σ$ is the standard deviation of the errors, and $\mathbb{L}^{(0)}_2, \mathbb{L}^{(1)}_2$ are universal random variables. This asymptotically pivotal LNE theory instantly yields a simple tuning-free procedure for constructing CIs with asymptotically exact coverage and optimal length for $f_0(x_0)$ and $f_0'(x_0)$. We also construct an asymptotically pivotal LNE for the anti-mode of $f_0$, and its limiting distribution does not even depend on $σ$. These asymptotically pivotal LNE theories are further extended to other convexity/concavity constrained models (e.g., log-concave density estimation) for which a limit distribution theory is available for problem-specific estimators.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

On a phase transition in general order spline regression

In the Gaussian sequence model $Y= θ_0 + \varepsilon$ in $\mathbb{R}^n$, we study the fundamental limit of approximating the signal $θ_0$ by a class $Θ(d,d_0,k)$ of (generalized) splines with free knots. Here $d$ is the degree of the spline, $d_0$ is the order of differentiability at each inner knot, and $k$ is the maximal number of pieces. We show that, given any integer $d\geq 0$ and $d_0\in\{-1,0,\ldots,d-1\}$, the minimax rate of estimation over $Θ(d,d_0,k)$ exhibits the following phase transition: \begin{equation*} \begin{aligned} \inf_{\widetildeθ}\sup_{θ\inΘ(d,d_0, k)}\mathbb{E}_θ\|\widetildeθ - θ\|^2 \asymp_d \begin{cases} k\log\log(16n/k), & 2\leq k\leq k_0,\\ k\log(en/k), & k \geq k_0+1. \end{cases} \end{aligned} \end{equation*} The transition boundary $k_0$, which takes the form $\lfloor{(d+1)/(d-d_0)\rfloor} + 1$, demonstrates the critical role of the regularity parameter $d_0$ in the separation between a faster $\log \log(16n)$ and a slower $\log(en)$ rate. We further show that, once encouraging an additional '$d$-monotonicity' shape constraint (including monotonicity for $d = 0$ and convexity for $d=1$), the above phase transition is eliminated and the faster $k\log\log(16n/k)$ rate can be achieved for all $k$. These results provide theoretical support for developing $\ell_0$-penalized (shape-constrained) spline regression procedures as useful alternatives to $\ell_1$- and $\ell_2$-penalized ones.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Multivariate convex regression: global risk bounds and adaptation

We study the problem of estimating a multivariate convex function defined on a convex body in a regression setting with random design. We are interested in optimal rates of convergence under a squared global continuous $l_2$ loss in the multivariate setting $(d\geq 2)$. One crucial fact is that the minimax risks depend heavily on the shape of the support of the regression function. It is shown that the global minimax risk is on the order of $n^{-2/(d+1)}$ when the support is sufficiently smooth, but that the rate $n^{-4/(d+4)}$ is when the support is a polytope. Such differences in rates are due to difficulties in estimating the regression function near the boundary of smooth regions. We then study the natural bounded least squares estimators (BLSE): we show that the BLSE nearly attains the optimal rates of convergence in low dimensions, while suffering rate-inefficiency in high dimensions. We show that the BLSE adapts nearly parametrically to polyhedral functions when the support is polyhedral in low dimensions by a local entropy method. We also show that the boundedness constraint cannot be dropped when risk is assessed via continuous $l_2$ loss. Given rate sub-optimality of the BLSE in higher dimensions, we further study rate-efficient adaptive estimation procedures. Two general model selection methods are developed to provide sieved adaptive estimators (SAE) that achieve nearly optimal rates of convergence for particular "regular" classes of convex functions, while maintaining nearly parametric rate-adaptivity to polyhedral functions in arbitrary dimensions. Interestingly, the uniform boundedness constraint is unnecessary when risks are measured in discrete $l_2$ norms.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Approximation and Estimation of s-Concave Densities via Rényi Divergences

In this paper, we study the approximation and estimation of $s$-concave densities via Rényi divergence. We first show that the approximation of a probability measure $Q$ by an $s$-concave densities exists and is unique via the procedure of minimizing a divergence functional proposed by Koenker and Mizera (2010) if and only if $Q$ admits full-dimensional support and a first moment. We also show continuity of the divergence functional in $Q$: if $Q_n \to Q$ in the Wasserstein metric, then the projected densities converge in weighted $L_1$ metrics and uniformly on closed subsets of the continuity set of the limit. Moreover, directional derivatives of the projected densities also enjoy local uniform convergence. This contains both on-the-model and off-the-model situations, and entails strong consistency of the divergence estimator of an $s$-concave density under mild conditions. One interesting and important feature for the Rényi divergence estimator of an $s$-concave density is that the estimator is intrinsically related with the estimation of log-concave densities via maximum likelihood methods. In fact, we show that for $d=1$ at least, the Rényi divergence estimators for $s$-concave densities converge to the maximum likelihood estimator of a log-concave density as $s \nearrow 0$. The Rényi divergence estimator shares similar characterizations as the MLE for log-concave distributions, which allows us to develop pointwise asymptotic distribution theory assuming that the underlying density is $s$-concave.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Relative Density and Exact Recovery in Heterogeneous Stochastic Block Models

The Stochastic Block Model (SBM) is a widely used random graph model for networks with communities. Despite the recent burst of interest in recovering communities in the SBM from statistical and computational points of view, there are still gaps in understanding the fundamental information theoretic and computational limits of recovery. In this paper, we consider the SBM in its full generality, where there is no restriction on the number and sizes of communities or how they grow with the number of nodes, as well as on the connection probabilities inside or across communities. This generality allows us to move past the artifacts of homogenous SBM, and understand the right parameters (such as the relative densities of communities) that define the various recovery thresholds. We outline the implications of our generalizations via a set of illustrative examples. For instance, $\log n$ is considered to be the standard lower bound on the cluster size for exact recovery via convex methods, for homogenous SBM. We show that it is possible, in the right circumstances (when sizes are spread and the smaller the cluster, the denser), to recover very small clusters (up to $\sqrt{\log n}$ size), if there are just a few of them (at most polylogarithmic in $n$).

0 citations0 reviewsNo code linkedOpen access