Some Properties of Rényi Entropy over Countably Infinite Alphabets

In this paper we study certain properties of Rényi entropy functionals $H_α(\mathcal{P})$ on the space of probability distributions over $\mathbb{Z}_+$. Primarily, continuity and convergence issues are addressed. Some properties shown parallel those known in the finite alphabet case, while others illustrate a quite different behaviour of Rényi entropy in the infinite case. In particular, it is shown that, for any distribution $\mathcal P$ and any $r\in[0,\infty]$, there exists a sequence of distributions $\mathcal{P}_n$ converging to $\mathcal{P}$ with respect to the total variation distance, such that $\lim_{n\to\infty}\lim_{α\to{1+}} H_α(\mathcal{P}_n) = \lim_{α\to{1+}}\lim_{n\to\infty} H_α(\mathcal{P}_n) + r$.

No reviews yet.

No comments yet.