Sampling two-dimensional spin systems with transformers
Autoregressive Neural Networks based on dense or convolutional layers have recently been shown to be a viable strategy for generating classical spin systems. Unlike these methods, sampling with transformers is commonly considered to be computationally inefficient. In this work, we propose a novel approach to transformer-based neural samplers in which we generate not a single spin per step but groups of spins. As an additional improvement, we construct a model of approximated probabilities, further improving the efficiency of the algorithm. Despite our approach being computationally heavier than dense networks or CNN-based approaches, we were able to sample larger systems of up to spins in case of the Ising model. The Effective Sample Size of our sampler is times larger than that of the previous state-of-the-art neural sampler when trained for the Ising model at critical temperature. Finally, we also test our algorithm on the 2D Edwards-Anderson model, where we train spin systems.