BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Daniel Polani

Daniel Polani appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Artificial IntelligenceMultiagent SystemsInformation Theorymath.ITMachine Learningnlin.AORoboticsComputer Science and Game TheoryHuman-Computer InteractionOther Quantitative Biologyphysics.app-phphysics.data-anPopulations and EvolutionSocial and Information Networks

Catalog footprint

What is connected

15works
14topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

15 published item(s)

preprint2022arXiv

Intrinsic Motivation in Dynamical Control Systems

Biological systems often choose actions without an explicit reward signal, a phenomenon known as intrinsic motivation. The computational principles underlying this behavior remain poorly understood. In this study, we investigate an information-theoretic approach to intrinsic motivation, based on maximizing an agent's empowerment (the mutual information between its past actions and future states). We show that this approach generalizes previous attempts to formalize intrinsic motivation, and we provide a computationally efficient algorithm for computing the necessary quantities. We test our approach on several benchmark control problems, and we explain its success in guiding intrinsically motivated behaviors by relating our information-theoretic control function to fundamental properties of the dynamical system representing the combined agent-environment system. This opens the door for designing practical artificial, intrinsically motivated controllers and for linking animal behaviors to their dynamical properties.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

AvE: Assistance via Empowerment

One difficulty in using artificial agents for human-assistive applications lies in the challenge of accurately assisting with a person's goal(s). Existing methods tend to rely on inferring the human's goal, which is challenging when there are many potential goals or when the set of candidate goals is difficult to identify. We propose a new paradigm for assistance by instead increasing the human's ability to control their environment, and formalize this approach by augmenting reinforcement learning with human empowerment. This task-agnostic objective preserves the person's autonomy and ability to achieve any eventual state. We test our approach against assistance based on goal inference, highlighting scenarios where our method overcomes failure modes stemming from goal ambiguity or misspecification. As existing methods for estimating empowerment in continuous domains are computationally hard, precluding its use in real time learned assistance, we also propose an efficient empowerment-inspired proxy metric. Using this, we are able to successfully demonstrate our method in a shared autonomy user study for a challenging simulated teleoperation task with human-in-the-loop training.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Human Perception of Intrinsically Motivated Autonomy in Human-Robot Interaction

A challenge in using robots in human-inhabited environments is to design behavior that is engaging, yet robust to the perturbations induced by human interaction. Our idea is to imbue the robot with intrinsic motivation (IM) so that it can handle new situations and appears as a genuine social other to humans and thus be of more interest to a human interaction partner. Human-robot interaction (HRI) experiments mainly focus on scripted or teleoperated robots, that mimic characteristics such as IM to control isolated behavior factors. This article presents a "robotologist" study design that allows comparing autonomously generated behaviors with each other, and, for the first time, evaluates the human perception of IM-based generated behavior in robots. We conducted a within-subjects user study (N=24) where participants interacted with a fully autonomous Sphero BB8 robot with different behavioral regimes: one realizing an adaptive, intrinsically motivated behavior and the other being reactive, but not adaptive. The robot and its behaviors are intentionally kept minimal to concentrate on the effect induced by IM. A quantitative analysis of post-interaction questionnaires showed a significantly higher perception of the dimension "Warmth" compared to the reactive baseline behavior. Warmth is considered a primary dimension for social attitude formation in human social cognition. A human perceived as warm (friendly, trustworthy) experiences more positive social interactions.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Cooperation and antagonism in information exchange in a growth scenario with two species

We consider a simple information-theoretic model of communication, in which two species of bacteria have the option of exchanging information about their environment, thereby improving their chances of survival. For this purpose, we model a system consisting of two species whose dynamics in the world are modelled by a bet-hedging strategy. It is well known that such models lend themselves to elegant information-theoretical interpretations by relating their respective long-term growth rate to the information the individual species has about its environment. We are specifically interested in modelling how this dynamics are affected when the species interact cooperatively or in an antagonistic way in a scenario with limited resources. For this purpose, we consider the exchange of environmental information between the two species in the framework of a game. Our results show that a transition from a cooperative to an antagonistic behaviour in a species results as a response to a change in the availability of resources. Species cooperate in abundance of resources, while they behave antagonistically in scarcity.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

An informational study of the evolution of codes and of emerging concepts in populations of agents

We consider the problem of the evolution of a code within a structured population of agents. The agents try to maximise their information about their environment by acquiring information from the outputs of other agents in the population. A naive use of information-theoretic methods would assume that every agent knows how to "interpret" the information offered by other agents. However, this assumes that one "knows" which other agents one observes, and thus which code they use. In our model, however, we wish to preclude that: it is not clear which other agents an agent is observing, and the resulting usable information is therefore influenced by the universality of the code used and by which agents an agent is "listening" to. We further investigate whether an agent who does not directly perceive the environment can distinguish states by observing other agents' outputs. For this purpose, we consider a population of different types of agents "talking" about different concepts, and try to extract new ones by considering their outputs only.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Informational parasites in code evolution

In a previous study, we considered an information-theoretic model of code evolution. In it, agents obtain information about their (common) environment by the perception of messages of other agents, which is determined by an interaction probability (the structure of the population). For an agent to understand another agent's messages, the former must either know the identity of the latter, or the code producing the messages must be universally interpretable. A universal code, however, introduces a vulnerability: a parasitic entity can take advantage of it. Here, we investigate this problem. In our specific setting, we consider a parasite to be an agent that tries to inflict as much damage as possible in the mutual understanding of the population (i.e. the parasite acts as a disinformation agent). We show that, after introducing a parasite in the population, the former adopts a code such that it captures the information about the environment that is missing in the population. Such agent would be of great value, but only if the rest of the population could understand its messages. However, it is of little use here, since the parasite utilises the most common messages in the population to express different concepts. Now we let the population respond by updating their codes such that, in this arms race, they again maximise their mutual understanding. As a result, there is a code drift in the population where the utilisation of the messages of the parasite is avoided. A consequence of this is that the information that the parasite possesses but the agents lack becomes understandable and readily available.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Changing the Environment Based on Empowerment as Intrinsic Motivation

One aspect of intelligence is the ability to restructure your own environment so that the world you live in becomes more beneficial to you. In this paper we investigate how the information-theoretic measure of agent empowerment can provide a task-independent, intrinsic motivation to restructure the world. We show how changes in embodiment and in the environment change the resulting behaviour of the agent and the artefacts left in the world. For this purpose, we introduce an approximation of the established empowerment formalism based on sparse sampling, which is simpler and significantly faster to compute for deterministic dynamics. Sparse sampling also introduces a degree of randomness into the decision making process, which turns out to beneficial for some cases. We then utilize the measure to generate agent behaviour for different agent embodiments in a Minecraft-inspired three dimensional block world. The paradigmatic results demonstrate that empowerment can be used as a suitable generic intrinsic motivation to not only generate actions in given static environments, as shown in the past, but also to modify existing environmental conditions. In doing so, the emerging strategies to modify an agent's environment turn out to be meaningful to the specific agent capabilities, i.e., de facto to its embodiment.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Don't Believe Everything You Hear; Preserving Relevant Information by Discarding Social Information

Integrating information gained by observing others via Social Bayesian Learning can be beneficial for an agent's performance, but can also enable population wide information cascades that perpetuate false beliefs through the agent population. We show how agents can influence the observation network by changing their probability of observing others, and demonstrate the existence of a population-wide equilibrium, where the advantages and disadvantages of the Social Bayesian update are balanced. We also use the formalism of relevant information to illustrate how negative information cascades are characterized by processing increasing amounts of non-relevant information.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Towards designing artificial universes for artificial agents under interaction closure

We are interested in designing artificial universes for artifi- cial agents. We view artificial agents as networks of high- level processes on top of of a low-level detailed-description system. We require that the high-level processes have some intrinsic explanatory power and we introduce an extension of informational closure namely interaction closure to capture this. Then we derive a method to design artificial universes in the form of finite Markov chains which exhibit high-level pro- cesses that satisfy the property of interaction closure. We also investigate control or information transfer which we see as an building block for networks representing artificial agents.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Changing the Environment based on Intrinsic Motivation

One of the remarkable feats of intelligent life is that it restructures the world it lives in for its own benefit. This extended abstract outlines how the information-theoretic principle of empowerment, as an intrinsic motivation, can be used to restructure the environment an agent lives in. We present a first qualitative evaluation of how an agent in a 3d-gridworld builds a staircase-like structure, which reflects the agent's embodiment.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Empowerment -- an Introduction

This book chapter is an introduction to and an overview of the information-theoretic, task independent utility function "Empowerment", which is defined as the channel capacity between an agent's actions and an agent's sensors. It quantifies how much influence and control an agent has over the world it can perceive. This book chapter discusses the general idea behind empowerment as an intrinsic motivation and showcases several previous applications of empowerment to demonstrate how empowerment can be applied to different sensor-motor configuration, and how the same formalism can lead to different observed behaviors. Furthermore, we also present a fast approximation for empowerment in the continuous domain.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

A Bivariate Measure of Redundant Information

We define a measure of redundant information based on projections in the space of probability distributions. Redundant information between random variables is information that is shared between those variables. But in contrast to mutual information, redundant information denotes information that is shared about the outcome of a third variable. Formalizing this concept, and being able to measure it, is required for the non-negative decomposition of mutual information into redundant and synergistic information. Previous attempts to formalize redundant or synergistic information struggle to capture some desired properties. We introduce a new formalism for redundant information and prove that it satisfies all the properties necessary outlined in earlier work, as well as an additional criterion that we propose to be necessary to capture redundancy. We also demonstrate the behaviour of this new measure for several examples, compare it to previous measures and apply it to the decomposition of transfer entropy.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Empowerment for Continuous Agent-Environment Systems

This paper develops generalizations of empowerment to continuous states. Empowerment is a recently introduced information-theoretic quantity motivated by hypotheses about the efficiency of the sensorimotor loop in biological organisms, but also from considerations stemming from curiosity-driven learning. Empowemerment measures, for agent-environment systems with stochastic transitions, how much influence an agent has on its environment, but only that influence that can be sensed by the agent sensors. It is an information-theoretic generalization of joint controllability (influence on environment) and observability (measurement by sensors) of the environment by the agent, both controllability and observability being usually defined in control theory as the dimensionality of the control/observation spaces. Earlier work has shown that empowerment has various interesting and relevant properties, e.g., it allows us to identify salient states using only the dynamics, and it can act as intrinsic reward without requiring an external reward. However, in this previous work empowerment was limited to the case of small-scale and discrete domains and furthermore state transition probabilities were assumed to be known. The goal of this paper is to extend empowerment to the significantly more important and relevant case of continuous vector-valued state spaces and initially unknown state transition probabilities. The continuous state space is addressed by Monte-Carlo approximation; the unknown transitions are addressed by model learning and prediction for which we apply Gaussian processes regression with iterated forecasting. In a number of well-known continuous control tasks we examine the dynamics induced by empowerment and include an application to exploration and online model learning.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Learning RoboCup-Keepaway with Kernels

We apply kernel-based methods to solve the difficult reinforcement learning problem of 3vs2 keepaway in RoboCup simulated soccer. Key challenges in keepaway are the high-dimensionality of the state space (rendering conventional discretization-based function approximation like tilecoding infeasible), the stochasticity due to noise and multiple learning agents needing to cooperate (meaning that the exact dynamics of the environment are unknown) and real-time learning (meaning that an efficient online implementation is required). We employ the general framework of approximate policy iteration with least-squares-based policy evaluation. As underlying function approximator we consider the family of regularization networks with subset of regressors approximation. The core of our proposed solution is an efficient recursive implementation with automatic supervised selection of relevant basis functions. Simulation results indicate that the behavior learned through our approach clearly outperforms the best results obtained earlier with tilecoding by Stone et al. (2005).

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Optimal strategies for fighting persistent bugs

Some microbial organisms are known to randomly slip into and out of hibernation, irrespective of environmental conditions [1]. In a (genetically) uniform population a typically very small subpopulation becomes metabolically inactive whereas the majority subpopulation remains active and grows. Bacteria such as E. coli, Staphylococcus aureus (MRSA-superbug), Mycobacterium tuberculosis, and Pseudomonas aeruginosa [1-3] show persistence. It can render bacteria less vulnerable in adverse environments [1, 4, 5] and their effective eradication through medication more difficult [2, 3, 6]. Here we show that medication treatment regimes may have to be modified when persistence is taken into account and characterize optimal approaches assuming that the total medication dose is constrained. The determining factors are cumulative toxicity, eradication power of the medication and bacterial response timescales. Persistent organisms have to be fought using tailored eradication strategies which display two fundamental characteristics. Ideally, the treatment time should be significantly longer than in the case of persistence with the medication uniformly spread out over time; however, if treatment time has to be limited, then the application of medication has to be concentrated towards the beginning and end of the treatment. These findings deviate from current clinical practice, and may therefore help to optimize and simplify treatments. Our use of multi-objective optimization [7] to map out the optimal strategies can be generalized to other related problems.

0 citations0 reviewsNo code linkedOpen access