BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Sarah Marzen

Sarah Marzen appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
cond-mat.stat-mechInformation Theorymath.ITnlin.CDBiological PhysicsMachine LearningArtificial IntelligenceComputational Engineering, Finance, and Sciencecond-mat.dis-nnmath.DSMolecular NetworksNeural and Evolutionary ComputingNeurons and Cognitionphysics.comp-phSubcellular Processes

Catalog footprint

What is connected

9works
15topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

9 published item(s)

preprint2025arXiv

New analytic formulae for memory and prediction functions in reservoir computers with time delays

Time delays increase the effective dimensionality of reservoirs, thus suggesting that time delays in reservoirs can enhance their performance, particularly their memory and prediction abilities. We find new closed-form expressions for memory and prediction functions of linear time-delayed reservoirs in terms of the power spectrum of the input and the reservoir transfer function. We confirm this relationship numerically for some time-delayed reservoirs using simulations, including when the reservoir can be linearized but is actually nonlinear. Finally, we use these closed-form formulae to address the utility of multiple time delays in linear reservoirs in order to perform memory and prediction, finding similar results to previous work on nonlinear reservoirs. We hope these closed-form formulae can be used to understand memory and predictive capabilities in time-delayed reservoirs.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

First-principles prediction of the information processing capacity of a simple genetic circuit

Given the stochastic nature of gene expression, genetically identical cells exposed to the same environmental inputs will produce different outputs. This heterogeneity has been hypothesized to have consequences for how cells are able to survive in changing environments. Recent work has explored the use of information theory as a framework to understand the accuracy with which cells can ascertain the state of their surroundings. Yet the predictive power of these approaches is limited and has not been rigorously tested using precision measurements. To that end, we generate a minimal model for a simple genetic circuit in which all parameter values for the model come from independently published data sets. We then predict the information processing capacity of the genetic circuit for a suite of biophysical parameters such as protein copy number and protein-DNA affinity. We compare these parameter-free predictions with an experimental determination of protein expression distributions and the resulting information processing capacity of E. coli cells. We find that our minimal model captures the scaling of the cell-to-cell variability in the data and the inferred information processing capacity of our simple genetic circuit up to a systematic deviation.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Quantifying many-body learning far from equilibrium with representation learning

Far-from-equilibrium many-body systems, from soap bubbles to suspensions to polymers, learn the drives that push them. This learning has been observed via thermodynamic properties, such as work absorption and strain. We move beyond these macroscopic properties that were first defined for equilibrium contexts: We quantify statistical mechanical learning with machine learning. Our toolkit relies on a structural parallel that we identify between far-from-equilibrium statistical mechanics and representation learning, which is undergone by neural networks that contain bottlenecks, including variational autoencoders. We train a variational autoencoder, via unsupervised learning, on configurations assumed by a many-body system during strong driving. We analyze the neural network's bottleneck to measure the many-body system's classification ability, memory capacity, discrimination ability, and novelty detection. Numerical simulations of a spin glass illustrate our technique. This toolkit exposes self-organization that eludes detection by thermodynamic measures, more reliably and more precisely identifying and quantifying learning by matter.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Memory and Information Processing in Recurrent Neural Networks

Recurrent neural networks (RNN) are simple dynamical systems whose computational power has been attributed to their short-term memory. Short-term memory of RNNs has been previously studied analytically only for the case of orthogonal networks, and only under annealed approximation, and uncorrelated input. Here for the first time, we present an exact solution to the memory capacity and the task-solving performance as a function of the structure of a given network instance, enabling direct determination of the function--structure relation in RNNs. We calculate the memory capacity for arbitrary networks with exponentially correlated input and further related it to the performance of the system on signal processing tasks in a supervised learning setup. We compute the expected error and the worst-case error bound as a function of the spectra of the network and the correlation structure of its inputs and outputs. Our results give an explanation for learning and generalization of task solving using short-term memory, which is crucial for building alternative computer architectures using physical phenomena based on the short-term memory principle.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Signatures of Infinity: Nonergodicity and Resource Scaling in Prediction, Complexity, and Learning

We introduce a simple analysis of the structural complexity of infinite-memory processes built from random samples of stationary, ergodic finite-memory component processes. Such processes are familiar from the well known multi-arm Bandit problem. We contrast our analysis with computation-theoretic and statistical inference approaches to understanding their complexity. The result is an alternative view of the relationship between predictability, complexity, and learning that highlights the distinct ways in which informational and correlational divergences arise in complex ergodic and nonergodic processes. We draw out consequences for the resource divergences that delineate the structural hierarchy of ergodic processes and for processes that are themselves hierarchical.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Circumventing the Curse of Dimensionality in Prediction: Causal Rate-Distortion for Infinite-Order Markov Processes

Predictive rate-distortion analysis suffers from the curse of dimensionality: clustering arbitrarily long pasts to retain information about arbitrarily long futures requires resources that typically grow exponentially with length. The challenge is compounded for infinite-order Markov processes, since conditioning on finite sequences cannot capture all of their past dependencies. Spectral arguments show that algorithms which cluster finite-length sequences fail dramatically when the underlying process has long-range temporal correlations and can fail even for processes generated by finite-memory hidden Markov models. We circumvent the curse of dimensionality in rate-distortion analysis of infinite-order processes by casting predictive rate-distortion objective functions in terms of the forward- and reverse-time causal states of computational mechanics. Examples demonstrate that the resulting causal rate-distortion theory substantially improves current predictive rate-distortion analyses.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Information Anatomy of Stochastic Equilibria

A stochastic nonlinear dynamical system generates information, as measured by its entropy rate. Some---the ephemeral information---is dissipated and some---the bound information---is actively stored and so affects future behavior. We derive analytic expressions for the ephemeral and bound informations in the limit of small-time discretization for two classical systems that exhibit dynamical equilibria: first-order Langevin equations (i) where the drift is the gradient of a potential function and the diffusion matrix is invertible and (ii) with a linear drift term (Ornstein-Uhlenbeck) but a noninvertible diffusion matrix. In both cases, the bound information is sensitive only to the drift, while the ephemeral information is sensitive only to the diffusion matrix and not to the drift. Notably, this information anatomy changes discontinuously as any of the diffusion coefficients vanishes, indicating that it is very sensitive to the noise structure. We then calculate the information anatomy of the stochastic cusp catastrophe and of particles diffusing in a heat bath in the overdamped limit, both examples of stochastic gradient descent on a potential landscape. Finally, we use our methods to calculate and compare approximations for the so-called time-local predictive information for adaptive agents.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Understanding and Designing Complex Systems: Response to "A framework for optimal high-level descriptions in science and engineering---preliminary report"

We recount recent history behind building compact models of nonlinear, complex processes and identifying their relevant macroscopic patterns or "macrostates". We give a synopsis of computational mechanics, predictive rate-distortion theory, and the role of information measures in monitoring model complexity and predictive performance. Computational mechanics provides a method to extract the optimal minimal predictive model for a given process. Rate-distortion theory provides methods for systematically approximating such models. We end by commenting on future prospects for developing a general framework that automatically discovers optimal compact models. As a response to the manuscript cited in the title above, this brief commentary corrects potentially misleading claims about its state space compression method and places it in a broader historical setting.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

An equivalence between a Maximum Caliber analysis of two-state kinetics and the Ising model

Application of the information-theoretic Maximum Caliber principle to the microtrajectories of a two-state system shows that the determination of key dynamical quantities can be mapped onto the evaluation of properties of the 1-D Ising model. The strategy described here is equivalent to an earlier Maximum Caliber formulation of the two-state problem, but reveals a different way of imposing the constraints which determine the probability distribution of allowed microtrajectories. The theoretical calculations of second moments, covariances, and correlation times that are obtained from Maximum Caliber agree well with simulated data of a particle diffusing on a double Gaussian surface, as well as with recent experiments on a particle trapped by a dual-well optical trap. The formalism reveals a new relationship between the average occupancy of the two states of the system, the average number of transitions between the two states that the system undergoes, Markov transition probabilities, and the discretization time step. In addition, Maxwell-like relations imply how measurements on one potential landscape can be used to make predictions about the dynamics on a different potential landscape, independent of further experiment.

0 citations0 reviewsNo code linkedOpen access