Paper detail

Correct ordering in the Zipf-Poisson ensemble

We consider a Zipf--Poisson ensemble in which $X_i\sim\poi(Ni^{-α})$ for $α>1$ and $N>0$ and integers $i\ge 1$. As $N\to\infty$ the first $n'(N)$ random variables have their proper order $X_1>X_2>...>X_{n'}$ relative to each other, with probability tending to 1 for $n'$ up to $(AN/\log(N))^{1/(α+2)}$ for an explicit constant $A(α)\ge 3/4$. The rate $N^{1/(α+2)}$ cannot be achieved. The ordering of the first $n'(N)$ entities does not preclude $X_m>X_{n'}$ for some interloping $m>n'$. The first $n"$ random variables are correctly ordered exclusive of any interlopers, with probability tending to 1 if $n"\le (BN/\log(N))^{1/(α+2)}$ for $B<A$. For a Zipf--Poisson model of the British National Corpus, which has a total word count of $100{,}000{,}000$, our result estimates that the 72 words with the highest counts are properly ordered.