Paper detail

Typical Sequences for Polish Alphabets

The notion of typical sequences plays a key role in the theory of information. Central to the idea of typicality is that a sequence $x_1, x_2, ..., x_n$ that is $P_X$-typical should, loosely speaking, have an empirical distribution that is in some sense close to the distribution $P_X$. The two most common notions of typicality are that of strong (letter) typicality and weak (entropy) typicality. While weak typicality allows one to apply many arguments that can be made with strongly typical arguments, some arguments for strong typicality cannot be generalized to weak typicality. In this paper, we consider an alternate definition of typicality, namely one based on the weak* topology and that is applicable to Polish alphabets (which includes $\reals^n$). This notion is a generalization of strong typicality in the sense that it degenerates to strong typicality in the finite alphabet case, and can also be applied to mixed and continuous distributions. Furthermore, it is strong enough to prove a Markov lemma, and thus can be used to directly prove a more general class of results than weak typicality. As an example of this technique, we directly prove achievability for Gel'fand-Pinsker channels with input constraints for a large class of alphabets and channels without first proving a finite alphabet result and then resorting to delicate quantization arguments. While this large class does not include Gaussian distributions with power constraints, it is shown to be straightforward to recover this case by considering a sequence of truncated Gaussian distributions.