BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Zhigang Bao

Zhigang Bao appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
math.PRmath.STStatistics Theorymath-phmath.MPMethodology

Catalog footprint

What is connected

22works
6topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

22 published item(s)

preprint2022arXiv

Non-splitting Neyman-Pearson Classifiers

The Neyman-Pearson (NP) binary classification paradigm constrains the more severe type of error (e.g., the type I error) under a preferred level while minimizing the other (e.g., the type II error). This paradigm is suitable for applications such as severe disease diagnosis, fraud detection, among others. A series of NP classifiers have been developed to guarantee the type I error control with high probability. However, these existing classifiers involve a sample splitting step: a mixture of class 0 and class 1 observations to construct a scoring function and some left-out class 0 observations to construct a threshold. This splitting enables classifier construction built upon independence, but it amounts to insufficient use of data for training and a potentially higher type II error. Leveraging a canonical linear discriminant analysis model, we derive a quantitative CLT for a certain functional of quadratic forms of the inverse of sample and population covariance matrices, and based on this result, develop for the first time NP classifiers without splitting the training sample. Numerical experiments have confirmed the advantages of our new non-splitting parametric strategy.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Spectral Statistics of Sample Block Correlation Matrices

A fundamental concept in multivariate statistics, sample correlation matrix, is often used to infer the correlation/dependence structure among random variables, when the population mean and covariance are unknown. A natural block extension of it, {\it sample block correlation matrix}, is proposed to take on the same role, when random variables are generalized to random sub-vectors. In this paper, we establish a spectral theory of the sample block correlation matrices and apply it to group independent test and related problem, under the high-dimensional setting. More specifically, we consider a random vector of dimension $p$, consisting of $k$ sub-vectors of dimension $p_t$'s, where $p_t$'s can vary from $1$ to order $p$. Our primary goal is to investigate the dependence of the $k$ sub-vectors. We construct a random matrix model called sample block correlation matrix based on $n$ samples for this purpose. The spectral statistics of the sample block correlation matrix include the classical Wilks' statistic and Schott's statistic as special cases. It turns out that the spectral statistics do not depend on the unknown population mean and covariance. Further, under the null hypothesis that the sub-vectors are independent, the limiting behavior of the spectral statistics can be described with the aid of the Free Probability Theory. Specifically, under three different settings of possibly $n$-dependent $k$ and $p_t$'s, we show that the empirical spectral distribution of the sample block correlation matrix converges to the free Poisson binomial distribution, free Poisson distribution (Marchenko-Pastur law) and free Gaussian distribution (semicircle law), respectively. We then further derive the CLTs for the linear spectral statistics of the block correlation matrix under general setting.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Central limit theorem for mesoscopic eigenvalue statistics of the free sum of matrices

We consider random matrices of the form $H_N=A_N+U_N B_N U^*_N$, where $A_N$, $B_N$ are two $N$ by $N$ deterministic Hermitian matrices and $U_N$ is a Haar distributed random unitary matrix. We establish a universal Central Limit Theorem for the linear eigenvalue statistics of $H_N$ on all mesoscopic scales inside the regular bulk of the spectrum. The proof is based on studying the characteristic function of the linear eigenvalue statistics, and consists of two main steps: (1) generating Ward identities using the left-translation-invariance of the Haar measure, along with a local law for the resolvent of $H_N$ and analytic subordination properties of the free additive convolution, allow us to derive an explicit formula for the derivative of the characteristic function; (2) a local law for two-point product functions of resolvents is derived using a partial randomness decomposition of the Haar measure. We also prove the corresponding results for orthogonal conjugations.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

On Cramér-von Mises statistic for the spectral distribution of random matrices

Let $F_N$ and $F$ be the empirical and limiting spectral distributions of an $N\times N$ Wigner matrix. The Cramér-von Mises (CvM) statistic is a classical goodness-of-fit statistic that characterizes the distance between $F_N$ and $F$ in $\ell^2$-norm. In this paper, we consider a mesoscopic approximation of the CvM statistic for Wigner matrices, and derive its limiting distribution. In the appendix, we also give the limiting distribution of the CvM statistic (without approximation) for the toy model CUE.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Singular vector and singular subspace distribution for the matrix denoising model

In this paper, we study the matrix denosing model $Y=S+X$, where $S$ is a low-rank deterministic signal matrix and $X$ is a random noise matrix, and both are $M\times n$. In the scenario that $M$ and $n$ are comparably large and the signals are supercritical, we study the fluctuation of the outlier singular vectors of $Y$. More specifically, we derive the limiting distribution of angles between the principal singular vectors of $Y$ and their deterministic counterparts, the singular vectors of $S$. Further, we also derive the distribution of the distance between the subspace spanned by the principal singular vectors of $Y$ and that spanned by the singular vectors of $S$. It turns out that the limiting distributions depend on the structure of the singular vectors of $S$ and the distribution of $X$, and thus they are non-universal.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Spectral rigidity for addition of random matrices at the regular edge

We consider the sum of two large Hermitian matrices $A$ and $B$ with a Haar unitary conjugation bringing them into a general relative position. We prove that the eigenvalue density on the scale slightly above the local eigenvalue spacing is asymptotically given by the free convolution of the laws of $A$ and $B$ as the dimension of the matrix increases. This implies optimal rigidity of the eigenvalues and optimal rate of convergence in Voiculescu's theorem. Our previous works [3,4] established these results in the bulk spectrum, the current paper completely settles the problem at the spectral edges provided they have the typical square-root behavior. The key element of our proof is to compensate the deterioration of the stability of the subordination equations by sharp error estimates that properly account for the local density near the edge. Our results also hold if the Haar unitary matrix is replaced by the Haar orthogonal matrix.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Statistical inference for principal components of spiked covariance matrices

In this paper, we study the asymptotic behavior of the extreme eigenvalues and eigenvectors of the high dimensional spiked sample covariance matrices, in the supercritical case when a reliable detection of spikes is possible. Especially, we derive the joint distribution of the extreme eigenvalues and the generalized components of the associated eigenvectors, i.e., the projections of the eigenvectors onto arbitrary given direction, assuming that the dimension and sample size are comparably large. In general, the joint distribution is given in terms of linear combinations of finitely many Gaussian and Chi-square variables, with parameters depending on the projection direction and the spikes. Our assumption on the spikes is fully general. First, the strengths of spikes are only required to be slightly above the critical threshold and no upper bound on the strengths is needed. Second, multiple spikes, i.e., spikes with the same strength, are allowed. Third, no structural assumption is imposed on the spikes. Thanks to the general setting, we can then apply the results to various high dimensional statistical hypothesis testing problems involving both the eigenvalues and eigenvectors. Specifically, we propose accurate and powerful statistics to conduct hypothesis testing on the principal components. These statistics are data-dependent and adaptive to the underlying true spikes. Numerical simulations also confirm the accuracy and powerfulness of our proposed statistics and illustrate significantly better performance compared to the existing methods in the literature. Especially, our methods are accurate and powerful even when either the spikes are small or the dimension is large.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Tracy-Widom limit for Kendall's tau

In this paper, we study a high-dimensional random matrix model from nonparametric statistics called the Kendall rank correlation matrix, which is a natural multivariate extension of the Kendall rank correlation coefficient. We establish the Tracy-Widom law for its largest eigenvalue. It is the first Tracy-Widom law for a nonparametric random matrix model, and also the first Tracy-Widom law for a high-dimensional U-statistic.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Local law of addition of random matrices on optimal scale

The eigenvalue distribution of the sum of two large Hermitian matrices, when one of them is conjugated by a Haar distributed unitary matrix, is asymptotically given by the free convolution of their spectral distributions. We prove that this convergence also holds locally in the bulk of the spectrum, down to the optimal scales larger than the eigenvalue spacing. The corresponding eigenvectors are fully delocalized. Similar results hold for the sum of two real symmetric matrices, when one is conjugated by a Haar orthogonal matrix.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Local Stability of the Free Additive Convolution

We prove that the system of subordination equations, defining the free additive convolution of two probability measures, is stable away from the edges of the support and blow-up singularities by showing that the recent smoothness condition of Kargin is always satisfied. As an application, we consider the local spectral statistics of the random matrix ensemble $A+UBU^*$, where $U$ is a Haar distributed random unitary or orthogonal matrix, and $A$ and $B$ are deterministic matrices. In the bulk regime, we prove that the empirical spectral distribution of $A+UBU^*$ concentrates around the free additive convolution of the spectral distributions of $A$ and $B$ on scales down to $N^{-2/3}$.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Delocalization for a class of random block band matrices

We consider $N\times N$ Hermitian random matrices $H$ consisting of blocks of size $M\geq N^{6/7}$. The matrix elements are i.i.d. within the blocks, close to a Gaussian in the four moment matching sense, but their distribution varies from block to block to form a block-band structure, with an essential band width $M$. We show that the entries of the Green's function $G(z)=(H-z)^{-1}$ satisfy the local semicircle law with spectral parameter $z=E+\mathbf{i}η$ down to the real axis for any $η\gg N^{-1}$, using a combination of the supersymmetry method inspired by \cite{Sh2014} and the Green's function comparison strategy. Previous estimates were valid only for $η\gg M^{-1}$. The new estimate also implies that the eigenvectors in the middle of the spectrum are fully delocalized.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Spectral statistics of large dimensional Spearman's rank correlation matrix and its application

Let $\mathbf{Q}=(Q_1,\ldots,Q_n)$ be a random vector drawn from the uniform distribution on the set of all $n!$ permutations of $\{1,2,\ldots,n\}$. Let $\mathbf{Z}=(Z_1,\ldots,Z_n)$, where $Z_j$ is the mean zero variance one random variable obtained by centralizing and normalizing $Q_j$, $j=1,\ldots,n$. Assume that $\mathbf {X}_i,i=1,\ldots ,p$ are i.i.d. copies of $\frac{1}{\sqrt{p}}\mathbf{Z}$ and $X=X_{p,n}$ is the $p\times n$ random matrix with $\mathbf{X}_i$ as its $i$th row. Then $S_n=XX^*$ is called the $p\times n$ Spearman's rank correlation matrix which can be regarded as a high dimensional extension of the classical nonparametric statistic Spearman's rank correlation coefficient between two independent random variables. In this paper, we establish a CLT for the linear spectral statistics of this nonparametric random matrix model in the scenario of high dimension, namely, $p=p(n)$ and $p/n\to c\in(0,\infty)$ as $n\to\infty$. We propose a novel evaluation scheme to estimate the core quantity in Anderson and Zeitouni's cumulant method in [Ann. Statist. 36 (2008) 2553-2576] to bypass the so-called joint cumulant summability. In addition, we raise a two-step comparison approach to obtain the explicit formulae for the mean and covariance functions in the CLT. Relying on this CLT, we then construct a distribution-free statistic to test complete independence for components of random vectors. Owing to the nonparametric property, we can use this test on generally distributed random variables including the heavy-tailed ones.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

The logarithmic law of random determinant

Consider the square random matrix $A_n=(a_{ij})_{n,n}$, where $\{a_{ij}:=a_{ij}^{(n)},i,j=1,\ldots,n\}$ is a collection of independent real random variables with means zero and variances one. Under the additional moment condition \[\sup_n\max_{1\leq i,j\leq n}\mathbb{E}a_{ij}^4<\infty,\] we prove Girko's logarithmic law of $\det A_n$ in the sense that as $n\rightarrow\infty$ \begin{eqnarray*}\frac{\log|\det A_n|-(1/2)\log(n-1)!}{\sqrt{(1/2)\log n}}\stackrel{d}{ \longrightarrow}N(0,1).\end{eqnarray*}

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Universality for the largest eigenvalue of sample covariance matrices with general population

This paper is aimed at deriving the universality of the largest eigenvalue of a class of high-dimensional real or complex sample covariance matrices of the form $\mathcal{W}_N=Σ^{1/2}XX^*Σ^{1/2}$. Here, $X=(x_{ij})_{M,N}$ is an $M\times N$ random matrix with independent entries $x_{ij},1\leq i\leq M,1\leq j\leq N$ such that $\mathbb{E}x_{ij}=0$, $\mathbb{E}|x_{ij}|^2=1/N$. On dimensionality, we assume that $M=M(N)$ and $N/M\rightarrow d\in(0,\infty)$ as $N\rightarrow\infty$. For a class of general deterministic positive-definite $M\times M$ matrices $Σ$, under some additional assumptions on the distribution of $x_{ij}$'s, we show that the limiting behavior of the largest eigenvalue of $\mathcal{W}_N$ is universal, via pursuing a Green function comparison strategy raised in [Probab. Theory Related Fields 154 (2012) 341-407, Adv. Math. 229 (2012) 1435-1515] by Erdős, Yau and Yin for Wigner matrices and extended by Pillai and Yin [Ann. Appl. Probab. 24 (2014) 935-1001] to sample covariance matrices in the null case ($Σ=I$). Consequently, in the standard complex case ($\mathbb{E}x_{ij}^2=0$), combing this universality property and the results known for Gaussian matrices obtained by El Karoui in [Ann. Probab. 35 (2007) 663-714] (nonsingular case) and Onatski in [Ann. Appl. Probab. 18 (2008) 470-490] (singular case), we show that after an appropriate normalization the largest eigenvalue of $\mathcal{W}_N$ converges weakly to the type 2 Tracy-Widom distribution $\mathrm{TW}_2$. Moreover, in the real case, we show that when $Σ$ is spiked with a fixed number of subcritical spikes, the type 1 Tracy-Widom limit $\mathrm{TW}_1$ holds for the normalized largest eigenvalue of $\mathcal {W}_N$, which extends a result of Féral and Péché in [J. Math. Phys. 50 (2009) 073302] to the scenario of nondiagonal $Σ$ and more generally distributed $X$.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Canonical correlation coefficients of high-dimensional normal vectors: finite rank case

Consider a normal vector $\mathbf{z}=(\mathbf{x}',\mathbf{y}')'$, consisting of two sub-vectors $\mathbf{x}$ and $\mathbf{y}$ with dimensions $p$ and $q$ respectively. With $n$ independent observations of $\mathbf{z}$ at hand, we study the correlation between $\mathbf{x}$ and $\mathbf{y}$, from the perspective of the Canonical Correlation Analysis, under the high-dimensional setting: both $p$ and $q$ are proportional to the sample size $n$. In this paper, we focus on the case that $Σ_{\mathbf{x}\mathbf{y}}$ is of finite rank $k$, i.e. there are $k$ nonzero canonical correlation coefficients, whose squares are denoted by $r_1\geq\cdots\geq r_k>0$. Under the additional assumptions $(p+q)/n\to y\in (0,1)$ and $p/q\not\to 1$, we study the sample counterparts of $r_i,i=1,\ldots,k$, i.e. the largest k eigenvalues of the sample canonical correlation matrix $S_{\mathbf{x}\mathbf{x}}^{-1}S_{\mathbf{x}\mathbf{y}}S_{\mathbf{y}\mathbf{y}}^{-1}S_{\mathbf{y}\mathbf{x}}$, namely $λ_1\geq\cdots\geq λ_k$. We show that there exists a threshold $r_c\in(0,1)$, such that for each $i\in\{1,\ldots,k\}$, when $r_i\leq r_c$, $λ_i$ converges almost surely to the right edge of the limiting spectral distribution of the sample canonical correlation matrix, denoted by $d_r$. When $r_i>r_c$, $λ_i$ possesses an almost sure limit in $(d_r,1]$, from which we can recover $r_i$ in turn, thus provide an estimate of the latter in the high-dimensional scenario.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Test of Independence for High-dimensional Random Vectors Based on Block Correlation Matrices

In this paper, we are concerned with the independence test for $k$ high-dimensional sub-vectors of a normal vector, with fixed positive integer $k$. A natural high-dimensional extension of the classical sample correlation matrix, namely block correlation matrix, is raised for this purpose. We then construct the so-called Schott type statistic as our test statistic, which turns out to be a particular linear spectral statistic of the block correlation matrix. Interestingly, the limiting behavior of the Schott type statistic can be figured out with the aid of the Free Probability Theory and the Random Matrix Theory. Specifically, we will bring the so-called real second order freeness for Haar distributed orthogonal matrices, derived in \cite{MP2013}, into the framework of this high-dimensional testing problem. Our test does not require the sample size to be larger than the total or any partial sum of the dimensions of the $k$ sub-vectors. Simulated results show the effect of the Schott type statistic, in contrast to those of the statistics proposed in \cite{JY2013} and \cite{JBZ2013}, is satisfactory. Real data analysis is also used to illustrate our method.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Universality for a global property of the eigenvectors of Wigner matrices

Let $M_n$ be an $n\times n$ real (resp. complex) Wigner matrix and $U_nΛ_n U_n^*$ be its spectral decomposition. Set $(y_1,y_2...,y_n)^T=U_n^*x$, where $x=(x_1,x_2,...,$ $x_n)^T$ is a real (resp. complex) unit vector. Under the assumption that the elements of $M_n$ have 4 matching moments with those of GOE (resp. GUE), we show that the process $X_n(t)=\sqrt{\frac{βn}{2}}\sum_{i=1}^{\lfloor nt\rfloor}(|y_i|^2-\frac1n)$ converges weakly to the Brownian bridge for any $\mathbf{x}$ such that $||x||_\infty\rightarrow 0$ as $n\rightarrow \infty$, where $β=1$ for the real case and $β=2$ for the complex case. Such a result indicates that the othorgonal (resp. unitary) matrices with columns being the eigenvectors of Wigner matrices are asymptotically Haar distributed on the orthorgonal (resp. unitary) group from a certain perspective.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Central limit theorem for partial linear eigenvalue statistics of Wigner matrices

In this paper, we study the complex Wigner matrices $M_n=\frac{1}{\sqrt{n}}W_n$ whose eigenvalues are typically in the interval $[-2,2]$. Let $λ_1\leq λ_2...\leqλ_n$ be the ordered eigenvalues of $M_n$. Under the assumption of four matching moments with the Gaussian Unitary Ensemble(GUE), for test function $f$ 4-times continuously differentiable on an open interval including $[-2,2]$, we establish central limit theorems for two types of partial linear statistics of the eigenvalues. The first type is defined with a threshold $u$ in the bulk of the Wigner semicircle law as $\mathcal{A}_n[f; u]=\sum_{l=1}^nf(λ_l)\mathbf{1}_{\{λ_l\leq u\}}$. And the second one is $\mathcal{B}_n[f; k]=\sum_{l=1}^{k}f(λ_l)$ with positive integer $k=k_n$ such that $k/n\rightarrow y\in (0,1)$ as $n$ tends to infinity. Moreover, we derive a weak convergence result for a partial sum process constructed from $\mathcal{B}_n[f; \lfloor nt\rfloor]$.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Local Semicircle law and Gaussian fluctuation for Hermite $β$ ensemble

Let $β>0$ and consider an $n$-point process $λ_1, λ_2,..., λ_n$ from Hermite $β$ ensemble on the real line $\mathbb{R}$. Dumitriu and Edelman discovered a tri-diagonal matrix model and established the global Wigner semicircle law for normalized empirical measures. In this paper we prove that the average number of states in a small interval in the bulk converges in probability when the length of the interval is larger than $\sqrt {\log n}$, i.e., local semicircle law holds. And the number of positive states in $(0,\infty)$ is proved to fluctuate normally around its mean $n/2$ with variance like $\log n/π^2β$. The proofs rely largely on the way invented by Valk$\acute{o}$ and Vir$\acute{a}$g of counting states in any interval and the classical martingale argument.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

On asymptotic expansion and CLT of linear eigenvalue statistics for sample covariance matrices when $N/M\rightarrow0$

We study the renormalized real sample covariance matrix $H=X^TX/\sqrt{MN}-\sqrt{M/N}$ with $N/M\rightarrow0$ as $N, M\rightarrow \infty$ in this paper. And we always assume $M=M(N)$. Here $X=[X_{jk}]_{M\times N}$ is an $M\times N$ real random matrix with i.i.d entries, and we assume $\mathbb{E}|X_{11}|^{5+δ}<\infty$ with some small positive $δ$. The Stieltjes transform $m_N(z)=N^{-1}Tr(H-z)^{-1}$ and the linear eigenvalue statistics of $H$ are considered. We mainly focus on the asymptotic expansion of $\mathbb{E}\{m_N(z)\}$ in this paper. Then for some fine test function, a central limit theorem for the linear eigenvalue statistics of $H$ is established. We show that the variance of the limiting normal distribution coincides with the case of a real Wigner matrix with Gaussian entries.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Tracy-Widom law for the extreme eigenvalues of sample correlation matrices

Let the sample correlation matrix be $W=YY^T$, where $Y=(y_{ij})_{p,n}$ with $y_{ij}=x_{ij}/\sqrt{\sum_{j=1}^nx_{ij}^2}$. We assume $\{x_{ij}: 1\leq i\leq p, 1\leq j\leq n\}$ to be a collection of independent symmetric distributed random variables with sub-exponential tails. Moreover, for any $i$, we assume $x_{ij}, 1\leq j\leq n$ to be identically distributed. We assume $0<p<n$ and $p/n\rightarrow y$ with some $y\in(0,1)$ as $p,n\rightarrow\infty$. In this paper, we provide the Tracy-Widom law ($TW_1$) for both the largest and smallest eigenvalues of $W$. If $x_{ij}$ are i.i.d. standard normal, we can derive the $TW_1$ for both the largest and smallest eigenvalues of the matrix $\mathcal{R}=RR^T$, where $R=(r_{ij})_{p,n}$ with $r_{ij}=(x_{ij}-\bar x_i)/\sqrt{\sum_{j=1}^n(x_{ij}-\bar x_i)^2}$, $\bar x_i=n^{-1}\sum_{j=1}^nx_{ij}$.

0 citations0 reviewsNo code linkedOpen access