Source author record

Aleksander Czechowski

Aleksander Czechowski appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

math.DS Machine Learning Artificial Intelligence Computer Science and Game Theory eess.SY General Literature math.AP math.GT math.OC math.SG Multiagent Systems Systems and Control

Catalog footprint

What is connected

6works

12topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2022arXiv

Poincaré-Bendixson Limit Sets in Multi-Agent Learning

A key challenge of evolutionary game theory and multi-agent learning is to characterize the limit behavior of game dynamics. Whereas convergence is often a property of learning algorithms in games satisfying a particular reward structure (e.g., zero-sum games), even basic learning models, such as the replicator dynamics, are not guaranteed to converge for general payoffs. Worse yet, chaotic behavior is possible even in rather simple games, such as variants of the Rock-Paper-Scissors game. Although chaotic behavior in learning dynamics can be precluded by the celebrated Poincaré-Bendixson theorem, it is only applicable to low-dimensional settings. Are there other characteristics of a game that can force regularity in the limit sets of learning? We show that behavior consistent with the Poincaré-Bendixson theorem (limit cycles, but no chaotic attractor) can follow purely from the topological structure of the interaction graph, even for high-dimensional settings with an arbitrary number of players and arbitrary payoff matrices. We prove our result for a wide class of follow-the-regularized leader (FoReL) dynamics, which generalize replicator dynamics, for binary games characterized interaction graphs where the payoffs of each player are only affected by one other player (i.e., interaction graphs of indegree one). Since chaos occurs already in games with only two players and three strategies, this class of non-chaotic games may be considered maximal. Moreover, we provide simple conditions under which such behavior translates into efficiency guarantees, implying that FoReL learning achieves time-averaged sum of payoffs at least as good as that of a Nash equilibrium, thereby connecting the topology of the dynamics to social-welfare analysis.

preprint2022arXiv

RangL: A Reinforcement Learning Competition Platform

The RangL project hosted by The Alan Turing Institute aims to encourage the wider uptake of reinforcement learning by supporting competitions relating to real-world dynamic decision problems. This article describes the reusable code repository developed by the RangL team and deployed for the 2022 Pathways to Net Zero Challenge, supported by the UK Net Zero Technology Centre. The winning solutions to this particular Challenge seek to optimize the UK's energy transition policy to net zero carbon emissions by 2050. The RangL repository includes an OpenAI Gym reinforcement learning environment and code that supports both submission to, and evaluation in, a remote instance of the open source EvalAI platform as well as all winning learning agent strategies. The repository is an illustrative example of RangL's capability to provide a reusable structure for future challenges.

preprint2021arXiv

Influence-aware Memory Architectures for Deep Reinforcement Learning

Due to its perceptual limitations, an agent may have too little information about the state of the environment to act optimally. In such cases, it is important to keep track of the observation history to uncover hidden state. Recent deep reinforcement learning methods use recurrent neural networks (RNN) to memorize past observations. However, these models are expensive to train and have convergence difficulties, especially when dealing with high dimensional input spaces. In this paper, we propose influence-aware memory (IAM), a theoretically inspired memory architecture that tries to alleviate the training difficulties by restricting the input of the recurrent layers to those variables that influence the hidden state information. Moreover, as opposed to standard RNNs, in which every piece of information used for estimating Q values is inevitably fed back into the network for the next prediction, our model allows information to flow without being necessarily stored in the RNN's internal memory. Results indicate that, by letting the recurrent layers focus on a small fraction of the observation variables while processing the rest of the information with a feedforward neural network, we can outperform standard recurrent architectures both in training speed and policy performance. This approach also reduces runtime and obtains better scores than methods that stack multiple observations to remove partial observability.

preprint2016arXiv

Existence of periodic solutions of the FitzHugh-Nagumo equations for an explicit range of the small parameter

The FitzHugh-Nagumo model describing propagation of nerve impulses in axon is given by fast-slow reaction-diffusion equations, with dependence on a parameter $ε$ representing the ratio of time scales. It is well known that for all sufficiently small $ε>0$ the system possesses a periodic traveling wave. With aid of computer-assisted rigorous computations, we prove the existence of this periodic orbit in the traveling wave equation for an explicit range $ε\in (0, 0.0015]$. Our approach is based on a novel method of combination of topological techniques of covering relations and isolating segments, for which we provide a self-contained theory. We show that the range of existence is wide enough, so the upper bound can be reached by standard validated continuation procedures. In particular, for the range $ε\in [1.5 \times 10^{-4}, 0.0015]$ we perform a rigorous continuation based on covering relations and not specifically tailored to the fast-slow setting. Moreover, we confirm that for $ε=0.0015$ the classical interval Newton-Moore method applied to a sequence of Poincaré maps already succeeds. Techniques described in this paper can be adapted to other fast-slow systems of similar structure.

preprint2016arXiv

Symplectomorphisms and discrete braid invariants

Area and orientation preserving diffeomorphisms of the standard 2-disc, referred to as symplectomorphisms of $\mathbb{D}^{2}$, allow decompositions in terms of positive twist diffeomorphisms. Using the latter decomposition we utilize the Conley index theory of discrete braid classes as introduced in [Ghrist et al., C. R. Acad. Sci. Paris Sér. I Math., 331(11), 2000, Invent. Math., 152(2), 2003] in order to obtain a Morse type forcing theory of periodic points: a priori information about periodic points determines a mapping class which may force additional periodic points.

preprint2015arXiv

Rigorous numerics for PDEs with indefinite tail: existence of a periodic solution of the Boussinesq equation with time-dependent forcing

We consider the Boussinesq PDE perturbed by a time-dependent forcing. Even though there is no smoothing effect for arbitrary smooth initial data, we are able to apply the method of self-consistent bounds to deduce the existence of smooth classical periodic solutions in the vicinity of 0. The proof is non-perturbative and relies on construction of periodic isolating segments in the Galerkin projections.