BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Hong Qian

Hong Qian appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
cond-mat.stat-mechmath-phmath.MPmath.PRphysics.chem-phBiological Physicsmath.DSSubcellular ProcessesPopulations and Evolutioncond-mat.mes-hallMachine Learningphysics.data-anArtificial IntelligenceMolecular Networksnlin.AOQuantitative MethodsBiomoleculesCell BehaviorComputation and Languagecond-mat.softNeural and Evolutionary Computingnlin.CDOther Quantitative Biologyphysics.class-phphysics.hist-phq-bio

Catalog footprint

What is connected

54works
26topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

54 published item(s)

preprint2022arXiv

Black-Box Tuning for Language-Model-as-a-Service

Extremely large pre-trained language models (PTMs) such as GPT-3 are usually released as a service. It allows users to design task-specific prompts to query the PTMs through some black-box APIs. In such a scenario, which we call Language-Model-as-a-Service (LMaaS), the gradients of PTMs are usually unavailable. Can we optimize the task prompts by only accessing the model inference APIs? This paper proposes the black-box tuning framework to optimize the continuous prompt prepended to the input text via derivative-free optimization. Instead of optimizing in the original high-dimensional prompt space, which is intractable for traditional derivative-free optimization, we perform optimization in a randomly generated subspace due to the low intrinsic dimensionality of large PTMs. The experimental results show that the black-box tuning with RoBERTa on a few labeled samples not only significantly outperforms manual prompt and GPT-3's in-context learning, but also surpasses the gradient-based counterparts, i.e., prompt tuning and full model tuning.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

On the Posterior Distribution of a Random Process Conditioned on Empirical Frequencies of a Finite Path: the i.i.d and finite Markov chain case

We obtain the posterior distribution of a random process conditioned on observing the empirical frequencies of a finite sample path. We find under a rather broad assumption on the "dependence structure" of the process, {\em c.f.} independence or Markovian, the posterior marginal distribution of the process at a given time index can be identified as certain empirical distribution computed from the observed empirical frequencies of the sample path. We show that in both cases of discrete-valued i.i.d. sequence and finite Markov chain, a certain "conditional symmetry" given by the observation of the empirical frequencies leads to the desired result on the posterior distribution. Results for both finite-time observations and its asymptotic infinite-time limit are connected via the idea of Gibbs conditioning. Finally, since our results demonstrate a central role of the empirical frequency in understanding the information content of data, we use the Large Deviations Principle (LDP) to construct a general notion of "data-driven entropy", from which one can apply a formalism from the recent study of statistical thermodynamics to data.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Unified Policy Optimization for Continuous-action Reinforcement Learning in Non-stationary Tasks and Games

This paper addresses policy learning in non-stationary environments and games with continuous actions. Rather than the classical reward maximization mechanism, inspired by the ideas of follow-the-regularized-leader (FTRL) and mirror descent (MD) update, we propose a no-regret style reinforcement learning algorithm PORL for continuous action tasks. We prove that PORL has a last-iterate convergence guarantee, which is important for adversarial and cooperative games. Empirical studies show that, in stationary environments such as MuJoCo locomotion controlling tasks, PORL performs equally well as, if not better than, the soft actor-critic (SAC) algorithm; in non-stationary environments including dynamical environments, adversarial training, and competitive games, PORL is superior to SAC in both a better final policy performance and a more stable training process.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Derivative-Free Reinforcement Learning: A Review

Reinforcement learning is about learning agent models that make the best sequential decisions in unknown environments. In an unknown environment, the agent needs to explore the environment while exploiting the collected information, which usually forms a sophisticated problem to solve. Derivative-free optimization, meanwhile, is capable of solving sophisticated problems. It commonly uses a sampling-and-updating framework to iteratively improve the solution, where exploration and exploitation are also needed to be well balanced. Therefore, derivative-free optimization deals with a similar core issue as reinforcement learning, and has been introduced in reinforcement learning approaches, under the names of learning classifier systems and neuroevolution/evolutionary reinforcement learning. Although such methods have been developed for decades, recently, derivative-free reinforcement learning exhibits attracting increasing attention. However, recent survey on this topic is still lacking. In this article, we summarize methods of derivative-free reinforcement learning to date, and organize the methods in aspects including parameter updating, model selection, exploration, and parallel/distributed methods. Moreover, we discuss some current limitations and possible future directions, hoping that this article could bring more attentions to this topic and serve as a catalyst for developing novel and efficient approaches.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Asymptotic Behavior of a Sequence of Conditional Probability Distributions and the Canonical Ensemble

The probability distribution of a function of a subsystem conditioned on the value of the function of the whole, in the limit when the ratio of their values goes to zero, has a limit law: It equals the unconditioned marginal probability distribution weighted by an exponential factor whose exponent is uniquely determined by the condition. We apply this theorem to explain the canonical equilibrium ensemble of a system in contact with a heat reservoir. Since the theorem only requires analysis at the level of the function of the subsystem and reservoir, it is applicable even without the knowledge of the composition of the reservoir itself, which extends the applicability of the canonical ensemble. Furthermore, we generalize our theorem to a model with strong interaction that contributes an additional term to the exponent, which is beyond the typical case of approximately additive functions. This result is new in both physics and mathematics, as a theory for the Gibbs conditioning principle for strongly correlated systems. A corollary provides a precise formulation of what a temperature bath is in probabilistic term

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Representations and Divergences in the Space of Probability Measures and Stochastic Thermodynamics

Radon-Nikodym (RN) derivative between two measures arises naturally in the affine structure of the space of probability measures with densities. Entropy, free energy, relative entropy, and entropy production as mathematical concepts associated with RN derivatives are introduced. We identify a simple equation that connects two measures with densities as a possible mathematical basis of the entropy balance equation that is central in nonequilibrium thermodynamics. Application of this formalism to Gibbsian canonical distribution yields many results in classical thermomechanics. An affine structure based on the canonical represenation and two divergences are introduced in the space of probability measures. It is shown that thermodynamic work, as a conditional expectation, is indictive of the RN derivative between two energy represenations being singular. The entropy divergence and the heat divergence yield respectively a Massieu-Planck potential based and a generalized Carnot inequalities.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

The Statistical Foundation of Entropy in Extended Irreversible Thermodynamics

In the theory of extended irreversible thermodynamics (EIT), the flux-dependent entropy function plays a key role and has a fundamental distinction from the usual flux-independent entropy function adopted by classical irreversible thermodynamics (CIT). However, its existence, as a prerequisite for EIT, and its statistical origin have never been justified. In this work, by studying the macroscopic limit of an ε-dependent Langevin dynamics, which admits a large deviations (LD) principle, we show that the stationary LD rate functions of probability density p_ε(x; t) and joint probability density p_ε(x; \dot{x}; t) actually turn out to be the desired fluxindependent entropy function in CIT and flux-dependent entropy function in EIT respectively. The difference of the two entropy functions is determined by the time resolution for Brownian motions times a Lagrangian, the latter arises from the LD Hamilton-Jacobi equation and can be used for constructing conserved Lagrangian/Hamiltonian dynamics.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Thermodynamics of Markov Processes with Non-extensive Entropy and Free Energy

Statistical thermodynamics of small systems shows dramatic differences from normal systems. Parallel to the recently presented steady-state thermodynamic formalism for master equation and Fokker-Planck equation, we show that a ``thermodynamic'' theory can also be developed based on Tsallis' generalized entropy $S^{(q)}=\sum_{i=1}^N(p_i-p_i^q)/[q(q-1)]$ and Shiino's generalized free energy $F^{(q)}=[\sum_{i=1}^Np_i(p_i/π_i)^{q-1}-1]/[q(q-1)]$, where $π_i$ is the stationary distribution. $dF^{(q)}/dt=-f_d^{(q)}\le 0$ and it is zero iff the system is in its stationary state. $dS^{(q)}/dt-Q_{ex}^{(q)} = f_d^{(q)}$ where $Q_{ex}^{(q)}$ characterizes the heat exchange. For systems approaching equilibrium with detailed balance, $f_d^{(q)}$ is the product of Onsager's thermodynamic flux and force. However, it is discovered that the Onsager's force is non-local. This is a consequence of the particular transformation invariance for zero energy of Tsallis' statistics.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Unified formalism for entropy productions and fluctuation relations

Stochastic entropy production, which quantifies the difference between the probabilities of trajectories of a stochastic dynamics and its time reversals, has a central role in nonequilibrium thermodynamics. In the theory of probability, the change in the statistical properties of observables can be represented by a change in the probability measure. We consider operators on the space of probability measure that induce changes in the statistical properties of a process, and formulate entropy productions in terms of these change-of-probability-measure (CPM) operators. This mathematical underpinning of the origin of entropy productions allows us to achieve an organization of various forms of fluctuation relations: All entropy productions have a non-negative mean value, admit the integral fluctuation theorem, and satisfy a rather general fluctuation relation. Other results such as the transient fluctuation theorem and detailed fluctuation theorems then are derived from the general fluctuation relation with more constraints on the operator. We use a discrete-time, discrete-state-space Markov process to draw the contradistinction among three reversals of a process: time reversal, protocol reversal and the dual process. The properties of their corresponding CPM operators are examined, and the domains of validity of various fluctuation relations for entropy productions in physics and chemistry are revealed. We also show that our CPM operator formalism can help us rather easily extend other fluctuations relations for excess work and heat, discuss the martingale properties of entropy productions, and derive the stochastic integral formulas for entropy productions in constant-noise diffusion process with Girsanov theorem. Our formalism provides a general and concise way to study the properties of entropy-related quantities in stochastic thermodynamics and information theory.

0 citations0 reviewsNo code linkedOpen access
preprint2019arXiv

Stochastic Dynamics II: Finite Random Dynamical Systems, Linear Representation, and Entropy Production

We study finite state random dynamical systems (RDS) and their induced Markov chains (MC) as stochastic models for complex dynamics. The linear representation of deterministic maps in RDS are matrix-valued random variables whose expectations correspond to the transition matrix of the MC. The instantaneous Gibbs entropy, Shannon-Khinchin entropy per step, and the entropy production rate of the MC are discussed. These three concepts as key anchor points in stochastic dynamics, characterize respectively the uncertainties of the system at instant time $t$, the randomness generated per step, and the dynamical asymmetry with respect to time reversal. The entropy production rate, expressed in terms of the cycle distributions, has found an expression in terms of the probability of the deterministic maps with the single attractor in the maximum entropy RDS. For finite RDS with invertible transformations, the non-negative entropy production rate of its MC is bounded above by the Kullback-Leibler divergence of the probability of the deterministic maps with respect to its time-reversal dual probability.

0 citations0 reviewsNo code linkedOpen access
preprint2019arXiv

Synchronization in discrete-time, discrete-state Random Dynamical Systems

We characterize synchronization phenomenon in discrete-time, discrete-state random dynamical systems, with random and probabilistic Boolean networks as particular examples. In terms of multiplicative ergodic properties of the induced linear cocycle, we show such a random dynamical system with finite state synchronizes if and only if the Lyapunov exponent $0$ has simple multiplicity. For the case of countable state space, characterization of synchronization is provided in terms of the spectral subspace corresponding to the Lyapunov exponent $-\infty$. In addition, for both cases of finite and countable state spaces, the mechanism of partial synchronization is described by partitioning the state set into synchronized subsets. Applications to biological networks are also discussed.

0 citations0 reviewsNo code linkedOpen access
preprint2019arXiv

Universal Relation Between Thermodynamic Driving Force and One-Way Fluxes in a Nonequilibrium Chemical Reaction with Complex Mechanism

In nonequilibrium chemical reaction systems, a fundamental relationship between unbalanced kinetic one-way fluxes and thermodynamic chemical driving forces is believed to exists. However this relation has been rigorously demonstrated only in a few cases in which one-way fluxes are well defined. In terms of its stochastic kinetic representation, we formulate the one-way fluxes for a general chemical reaction far from equilibrium, with arbitrary complex mechanisms, multiple intermediates, and internal kinetic cycles. For each kinetic cycle, the logarithm of the ratio of the steady-state forward and backward one-way fluxes is equal to the free energy difference between the reactants and products along the cycle. This fundamental relation is further established for general chemical reaction networks with multiple input and output complexes. Our result not only provides an equivalent definition of free energy difference in nonequilibrium chemical reaction networks, it also unifies the stochastic and macroscopic nonequilibrium chemical thermodynamics in a very broad sense.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

A Discrete Stochastic Formulation for Reversible Bimolecular Reactions via Diffusion Encounter

The classical models for irreversible diffusion-influenced reactions can be derived by introducing absorbing boundary conditions to over-damped continuous Brownian motion (BM) theory. As there is a clear corresponding stochastic process, the mathematical description takes both Kolmogorov forward equation for the evolution of the probability distribution function and the stochastic sample trajectories. This dual description is a fundamental characteristic of stochastic processes and allows simple particle based simulations to accurately match the expected statistical behavior. However, in the traditional theory using the back-reaction boundary condition to model reversible reactions with geminate recombinations, several subtleties arise: it is unclear what the underlying stochastic process is, which causes complications in producing accurate simulations; and it is non-trivial how to perform an appropriate discretization for numerical computations. In this work, we derive a discrete stochastic model that recovers the classical models and their boundary conditions in the continuous limit. In the case of reversible reactions, we recover the back-reaction boundary condition, unifying the back-reaction approach with those of current simulation packages. Furthermore, all the complications encountered in the continuous models become trivial in the discrete model. Our formulation brings to attention the question: With computations in mind, can we develop a discrete reaction kinetics model that is more fundamental than its continuous counterpart?

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

A framework towards understanding mesoscopic phenomena: Emergent unpredictability, symmetry breaking and dynamics across scales

By integrating 4 lines of thoughts: symmetry breaking originally advanced by Anderson, bifurcation from nonlinear dynamics, Landau's theory of phase transition, and the mechanism of emergent rare events studied by Kramers, we introduce a possible framework for understanding mesoscopic dynamics that links (i) fast lower level microscopic motions, (ii) movements within each basin at the mid-level, and (iii) higher-level rare transitions between neighboring basins, which have rates that decrease exponentially with the size of the system. In this mesoscopic framework, multiple attractors arise as emergent properties of the nonlinear systems. The interplay between the stochasticity and nonlinearity leads to successive jump-like transitions among different basins. We argue each transition is a dynamic symmetry breaking, with the potential of exhibiting Thom-Zeeman catastrophe as well as phase transition with the breakdown of ergodicity (e.g., cell differentiation). The slow-time dynamics of the nonlinear mesoscopic system is not deterministic, rather it is a discrete stochastic jump process. The existence of these discrete states and the Markov transitions among them are both emergent phenomena. This emergent stochastic jump dynamics then serves as the stochastic element for the nonlinear dynamics of a higher level aggregates on an even larger spatial and slower time scales (e.g., evolution). This description captures the hierarchical structure outlined by Anderson and illustrates two distinct types of limit of a mesoscopic dynamics: A long-time ensemble thermodynamics in terms of time $t$ tending infinity followed by the size of the system $N$ tending infinity, and a short-time trajectory steady state with $N$ tending infinity followed by $t$ tending infinity. With these limits, symmetry breaking and cusp catastrophe are two perspectives of the same mesoscopic system on different time scales.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Entropy Production in Mesoscopic Stochastic Thermodynamics: Nonequilibrium Kinetic Cycles Driven by Chemical Potentials, Temperatures, and Mechanical Forces

Nonequilibrium thermodynamics (NET) investigates processes in systems out of global equilibrium. On a mesoscopic level, it provides a statistical dynamic description of various complex phenomena such as chemical reactions, ion transport, diffusion, thermochemical, thermomechanical and mechanochemical fluxes. In the present review, we introduce a mesoscopic stochastic formulation of NET by analyzing entropy production in several simple examples. The fundamental role of nonequilibrium steady-state cycle kinetics is emphasized. The statistical mechanics of Onsager's reciprocal relations in this context is elucidated. Chemomechanical, thermomechanical, and enzyme-catalyzed thermochemical energy transduction processes are discussed. It is argued that mesoscopic stochastic NET provides a rigorous mathematical basis of fundamental concepts needed for understanding complex processes in chemistry, physics and biology, and which is also relevant for nanoscale technological advances.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Mathematical Formalism of Nonequilibrium Thermodynamics for Nonlinear Chemical Reaction Systems with General Rate Law

This paper studies a mathematical formalism of nonequilibrium thermodynamics for chemical reaction models with $N$ species, $M$ reactions, and general rate law. We establish a mathematical basis for J. W. Gibbs' macroscopic chemical thermodynamics under G. N. Lewis' kinetic law of entire equilibrium (detailed balance in nonlinear chemistry kinetics). In doing so, the equilibrium thermodynamics is then naturally generalized to nonequilibrium settings without detailed balance. The kinetic models are represented by a Markovian jumping process. A generalized macroscopic chemical free energy function and its associated balance equation with nonnegative source and sink are the major discoveries. The proof is based on the large deviation principle of this type of Markov processes. A general fluctuation dissipation theorem for stochastic reaction kinetics is also proved. The mathematical theory illustrates how a novel macroscopic dynamic law can emerges from the mesoscopic kinetics in a multi-scale system.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Mesoscopic Kinetic Basis of Macroscopic Chemical Thermodynamics: A Mathematical Theory

From a mathematical model that describes a complex chemical kinetic system of $N$ species and $M$ elementrary reactions in a rapidly stirred vessel of size $V$ as a Markov process, we show that a macroscopic chemical thermodynamics emerges as $V\rightarrow\infty$. The theory is applicable to linear and nonlinear reactions, closed systems reaching chemical equilibrium, or open, driven systems approaching to nonequilibrium steady states. A generalized mesoscopic free energy gives rise to a macroscopic chemical energy function $φ^{ss}(\vx)$ where $\vx=(x_1,\cdots,x_N)$ are the concentrations of the $N$ chemical species. The macroscopic chemical dynamics $\vx(t)$ satisfies two emergent laws: (1) $(\rd/\rd t)φ^{ss}[\vx(t)]\le 0$, and (2)$(\rd/\rd t)φ^{ss}[\vx(t)]=\text{cmf}(\vx)-σ(\vx)$ where entropy production rate $σ\ge 0$ represents the sink for the chemical energy, and chemical motive force $\text{cmf}\ge 0$ is non-zero if the system is driven under a sustained nonequilibrium chemostat. For systems with detailed balance $\text{cmf}=0$, and if one assumes the law of mass action,$φ^{ss}(\vx)$ is precisely the Gibbs' function $\sum_{i=1}^N x_i\big[μ_i^o+\ln x_i\big]$ for ideal solutions. For a class of kinetic systems called complex balanced, which include many nonlinear systems as well as many simple open, driven chemical systems, the $φ^{ss}(\vx)$, with global minimum at $\vx^*$, has the generic form $\sum_{i=1}^N x_i\big[\ln(x_i/x_i^*)-x_i+x_i^*\big]$,which has been known in chemical kinetic literature.Macroscopic emergent "laws" are independent of the details of the underlying kinetics. This theory provides a concrete example from chemistry showing how a dynamic macroscopic law can emerge from the kinetics at a level below.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Nonequilibrium Thermodynamic Formalism of Nonlinear Chemical Reaction Systems with Waage-Guldberg's Law of Mass Action

Macroscopic entropy production $σ^{(tot)}$ in the general nonlinear isothermal chemical reaction system with mass action kinetics is decomposed into a free energy dissipation and a house-keeping heat: $σ^{(tot)}=σ^{(fd)}+σ^{(hk)}$; $σ^{(fd)}=-\rd A/\rd t$, where $A$ is a generalized free energy function. This yields a novel nonequilibrium free energy balance equation $\rd A/\rd t=-σ^{(tot)}+σ^{(hk)}$, which is on a par with celebrated entropy balance equation $\rd S/\rd t=σ^{(tot)}+η^{(ex)}$ where $η^{(ex)}$ is the rate of entropy exchange with the environment.For kinetic systems with complex balance, $σ^{(fd)}$ and $σ^{(hk)}$ are the macroscopic limits of stochastic free energy dissipation and house-keeping heat, which are both nonnegative, in the Delbrück-Gillespie description of the stochastic chemical kinetics.Therefore, we show that a full kinetic and thermodynamic theory of chemical reaction systems that transcends mesoscopic and macroscopic levels emerges.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Nonlinear Stochastic Dynamics of Complex Systems, II: Potential of Entropic Force in Markov Systems with Nonequilibrium Steady State, Generalized Gibbs Function and Criticality

In this paper we revisit the notion of the "minus logarithm of stationary probability" as a generalized potential in nonequilibrium systems and attempt to illustrate its central role in an axiomatic approach to stochastic nonequilibrium thermodynamics of complex systems. It is demonstrated that this quantity arises naturally through both monotonicity results of Markov processes and as the rate function when a stochastic process approaches a deterministic limit. We then undertake a more detailed mathematical analysis of the consequences of this quantity, culminating in a necessary and sufficient condition for the criticality of stochastic systems. This condition is then discussed in the context of recent results about criticality in biological systems

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

A Thermodynamic Theory of Ecology: Helmholtz Theorem for Lotka-Volterra Equation, Extended Conservation Law, and Stochastic Predator-Prey Dynamics

We carry out mathematical analyses, {\em à la} Helmholtz's and Boltzmann's 1884 studies of monocyclic Newtonian dynamics, for the Lotka-Volterra (LV) equation exhibiting predator-prey oscillations. In doing so a novel "thermodynamic theory" of ecology is introduced. An important feature, absent in the classical mechanics, of ecological systems is a natural stochastic population dynamic formulation of which the deterministic equation (e.g., the LV equation studied) is the infinite population limit. Invariant density for the stochastic dynamics plays a central role in the deterministic LV dynamics. We show how the conservation law along a single trajectory extends to incorporate both variations in a model parameter $α$ and in initial conditions: Helmholtz's theorem establishes a broadly valid conservation law in a class of ecological dynamics. We analyze the relationships among mean ecological activeness $θ$, quantities characterizing dynamic ranges of populations $\mathcal{A}$ and $α$, and the ecological force $F_α$. The analyses identify an entire orbit as a stationary ecology, and establish the notion of "equation of ecological states". Studies of the stochastic dynamics with finite populations show the LV equation as the robust, fast cyclic underlying behavior. The mathematical narrative provides a novel way of capturing long-term dynamical behaviors with an emergent {\em conservative ecology}.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Stochastic Free Energies, Conditional Probability and Legendre Transform for Ensemble Change

This work extends a recently developed mathematical theory of thermodynamics for Markov processes with, and more importantly, without detailed balance. We show that the Legendre transform in connection to ensemble changes in Gibbs' statistical mechanics can be derived from the stochastic theory. We consider the joint probability $p_{XY}$ of two random variables X and Y and the conditional probability $p_{X|Y=y^*}$, with $y^*=<Y>$ according to $p_{XY}$. The stochastic free energies of the XY system (fluctuating Y ensemble) and the $X|Y$ system (fixed Y ensemble) are related by the chain rule for relative entropy. In the thermodynamic limit, defined as $V,Y\to\infty$ (where one assumes Y as an extensive quantity, while V denotes the system size parameter), the marginal probability obeys $p_Y(y)\to \exp(-Vϕ(z))$ with $z=y/V$. A conjugate variable $μ=-ϕ(z)/z$ naturally emerges from this result. The stochastic free energies of the fluctuating and fixed ensembles are then related by $F_{XY}=F_{X|Y=y^*}-μy^*$, with $μ=\partial F_{X|Y=y}/\partial y$. The time evolutions for the two free energies are the same: $d[F_{XY}(t)]/dt=\partial [F_{X|Y=y^*(t)}(t)]/ \partial t$. This mathematical theory is applied to systems with fixed and fluctuating number of identical independent random variables (idea gas), as well as to microcanonical systems with uniform stationary probability.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Stochastic phenotype transition of a single cell in an intermediate region of gene-state switching

Multiple phenotypic states often arise in a single cell with different gene-expression states that undergo transcription regulation with positive feedback. Recent experiments have shown that at least in E. coli, the gene state switching can be neither extremely slow nor exceedingly rapid as many previous theoretical treatments assumed. Rather it is in the intermediate region which is difficult to handle mathematically.Under this condition, from a full chemical-master-equation description we derive a model in which the protein copy-number, for a given gene state, follow a deterministic mean-field description while the protein synthesis rates fluctuate due to stochastic gene-state switching. The simplified kinetics yields a nonequilibrium landscape function, which, similar to the energy function for equilibrium fluctuation, provides the leading orders of fluctuations around each phenotypic state, as well as the transition rates between the two phenotypic states. This rate formula is analogous to Kramers theory for chemical reactions. The resulting behaviors are significantly different from the two limiting cases studied previously.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Thermodynamics of the General Diffusion Process: Equilibrium Supercurrent and Nonequilibrium Driven Circulation with Dissipation

Unbalanced probability circulation, which yields cyclic motions in phase space, is the defining characteristics of a stationary diffusion process without detailed balance. In over-damped soft matter systems, such behavior is a hallmark of the presence of a sustained external driving force accompanied with dissipations. In an under-damped and strongly correlated system, however, cyclic motions are often the consequences of a conservative dynamics. In the present paper, we give a novel interpretation of a class of diffusion processes with stationary circulation in terms of a Maxwell-Boltzmann equilibrium in which cyclic motions are on the level set of stationary probability density function thus non-dissipative, e.g., a supercurrent. This implies an orthogonality between stationary circulation $J^{ss}(x)$ and the gradient of stationary probability density $f^{ss}(x)>0$. A sufficient and necessary condition for the orthogonality is a decomposition of the drift $b(x)=j(x)+ D(x)\nablaφ(x)$ where $\nabla\cdot j(x)=0$ and $j(x)$ $\cdot\nablaφ(x)=0$. Stationary processes with Maxwell-Boltzmann equilibrium has an underlying conservative dynamics $\dot{x}= j(x)\equiv$ $\big(f^{ss}(x)\big)^{-1}J^{ss}(x)$, and a first integral $φ(x)\equiv-\ln f^{ss}(x)=$ const, akin to a Hamiltonian system. At all time, an instantaneous free energy balance equation exists for a given diffusion system; and an extended energy conservation law among a family of diffusion processes with different parameter $α$ can be established via a Helmholtz theorem. For the general diffusion process without the orthogonality, a nonequilibrium cycle emerges, which consists of external driven $φ$-ascending steps and spontaneous $φ$-descending movements, alternated with iso-$φ$ motions. The theory presented here provides a rich mathematical narrative for complex mesoscopic dynamics.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Universal Ideal Behavior and Macroscopic Work Relation of Linear Irreversible Stochastic Thermodynamics

We revisit the Ornstein-Uhlenbeck (OU) process as the fundamental mathematical description of linear irreversible phenomena, with fluctuations, near an equilibrium. By identifying the underlying circulating dynamics in a stationary process as the natural generalization of classical conservative mechanics, a bridge between a family of OU processes with equilibrium fluctuations and thermodynamics is established through the celebrated Helmholtz theorem. The Helmholtz theorem provides an emergent macroscopic "equation of state" of the entire system, which exhibits a universal ideal thermodynamic behavior. Fluctuating macroscopic quantities are studied from the stochastic thermodynamic point of view and a non-equilibrium work relation is obtained in the macroscopic picture, which may facilitate experimental study and application of the equalities due to Jarzynski, Crooks, and Hatano and Sasa.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Fitness and entropy production in a cell population dynamics with epigenetic phenotype switching

Motivated by recent understandings in the stochastic natures of gene expression, biochemical signaling, and spontaneous reversible epigenetic switchings, we study a simple deterministic cell population dynamics in which subpopulations grow with different rates and individual cells can bi-directionally switch between a small number of different epigenetic phenotypes. Two theories in the past, the population dynamics and thermodynamics of master equations, separatedly defined two important concepts in mathematical terms: the {\em fitness} in the former and the (non-adiabatic) {\em entropy production} in the latter. Both play important roles in the evolution of the cell population dynamics. The switching sustains the variations among the subpopulation growth thus continuous natural selection. As a form of Price's equation, the fitness increases with ($i$) natural selection through variations and $(ii)$ a positive covariance between the per capita growth and switching, which represents a Lamarchian-like behavior. A negative covariance balances the natural selection in a fitness steady state | "the red queen" scenario. At the same time the growth keeps the proportions of subpopulations away from the "intrinsic" switching equilibrium of individual cells, thus leads to a continous entropy production. A covariance, between the per capita growth rate and the "chemical potential" of subpopulation, counter-acts the entropy production. Analytical results are obtained for the limiting cases of growth dominating switching and vice versa.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Fluorescence Correlation Spectroscopy and Nonlinear Stochastic Reaction-Diffusion

The currently existing theory of fluorescence correlation spectroscopy(FCS) is based on the linear fluctuation theory originally developed by Einstein, Onsager, Lax, and others as a phenomenological approach to equilibrium fluctuations in bulk solutions. For mesoscopic reaction-diffusion systems with nonlinear chemical reactions among a small number of molecules, a situation often encountered in single-cell biochemistry, it is expected that FCS time correlation functions of a reaction-diffusion system can deviate from the classic results of Elson and Magde. We first discuss this nonlinear effect for reaction systems without diffusion. For nonlinear stochastic reaction-diffusion systems here are no closed solutions; therefore, stochastic Monte-Carlo simulations are carried out. We show that the deviation is small for a simple bimolecular reaction; the most significant deviations occur when the number of molecules is small and of the same order. Our results show that current linear FCS theory could be adequate for measurements on biological systems that contain many other sources of uncertainties. At the same time it provides a framework for future measurements of nonlinear, fluctuating chemical reactions with high-precision FCS. Extending Delbrück-Gillespie's theory for stochastic nonlinear reactions with rapidly stirring to reaction-diffusion systems provides a mesoscopic model for chemical and biochemical reactions at nanometric and mesoscopic level such as a single biological cell.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Small Open Chemical Systems Theory and Its Implications to Darwinian Evolutionary Dynamics, Complex Self-Organization and Beyond

The study of biological cells in terms of mesoscopic, nonequilibrium, nonlinear, stochastic dynamics of open chemical systems provides a paradigm for other complex, self-organizing systems with ultra-fast stochastic fluctuations, short-time deterministic nonlinear dynamics, and long-time evolutionary behavior with exponentially distributed rare events, discrete jumps among punctuated equilibria, and catastrophe.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Stochastic Dynamics of Electrical Membrane with Voltage-Dependent Ion Channel Fluctuations

Brownian ratchet like stochastic theory for the electrochemical membrane system of Hodgkin-Huxley (HH) is developed. The system is characterized by a continuous variable $Q_m(t)$, representing mobile membrane charge density, and a discrete variable $K_t$ representing ion channel conformational dynamics. A Nernst-Planck-Nyquist-Johnson type equilibrium is obtained when multiple conducting ions have a common reversal potential. Detailed balance yields a previously unknown relation between the channel switching rates and membrane capacitance, bypassing Eyring-type explicit treatment of gating charge kinetics. From a molecular structural standpoint, membrane charge $Q_m$ is a more natural dynamic variable than potential $V_m$; our formalism treats $Q_m$-dependent conformational transition rates $λ_{ij}$ as intrinsic parameters. Therefore in principle, $λ_{ij}$ vs. $V_m$ is experimental protocol dependent,e.g., different from voltage or charge clamping measurements. For constant membrane capacitance per unit area $C_m$ and neglecting membrane potential induced by gating charges, $V_m=Q_m/C_m$, and HH's formalism is recovered. The presence of two types of ions, with different channels and reversal potentials, gives rise to a nonequilibrium steady state with positive entropy production $e_p$. For rapidly fluctuating channels, an expression for $e_p$ is obtained.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

The Sampling-and-Learning Framework: A Statistical View of Evolutionary Algorithms

Evolutionary algorithms (EAs), a large class of general purpose optimization algorithms inspired from the natural phenomena, are widely used in various industrial optimizations and often show excellent performance. This paper presents an attempt towards revealing their general power from a statistical view of EAs. By summarizing a large range of EAs into the sampling-and-learning framework, we show that the framework directly admits a general analysis on the probable-absolute-approximate (PAA) query complexity. We particularly focus on the framework with the learning subroutine being restricted as a binary classification, which results in the sampling-and-classification (SAC) algorithms. With the help of the learning theory, we obtain a general upper bound on the PAA query complexity of SAC algorithms. We further compare SAC algorithms with the uniform search in different situations. Under the error-target independence condition, we show that SAC algorithms can achieve polynomial speedup to the uniform search, but not super-polynomial speedup. Under the one-side-error condition, we show that super-polynomial speedup can be achieved. This work only touches the surface of the framework. Its power under other conditions is still open.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

A Decomposition of Irreversible Diffusion Processes Without Detailed Balance

As a generalization of deterministic, nonlinear conservative dynamical systems, a notion of {\em canonical conservative dynamics} with respect to a positive, differentiable stationary density $ρ(x)$ is introduced: $\dot{x}=j(x)$ in which $\nabla\cdot\big(ρ(x)j(x)\big)=0$. Such systems have a conserved "generalized free energy function" $F[u]$ $=$ $\int u(x,t)\ln\big(u(x,t)/ρ(x)\big)dx$ in phase space with a density flow $u(x,t)$ satisfying $\partial u_t =-\nabla\cdot(ju)$. Any general stochastic diffusion process without detailed balance, in terms of its Fokker-Planck equation, can be decomposed into a reversible diffusion process with detailed balance and a canonical conservative dynamics. This decomposition can be rigorously established in a function space with inner product defined as $\langleϕ,ψ\rangle=\intρ^{-1}(x)ϕ(x)ψ(x)dx$. Furthermore, a law for balancing $F[u]$ can be obtained: The non-positive $dF[u(x,t)]/dt$ $=$ $E_{in}(t)-e_p(t)$ where the "source" $E_{in}(t)\ge 0$ and the "sink" $e_p(t)\ge 0$ are known as house-keeping heat and entropy production, respectively. A reversible diffusion has $E_{in}(t)=0$. For a linear (Ornstein-Uhlenbeck) diffusion process, our decomposition is equivalent to the previous approaches developed by R. Graham and P. Ao, as well as the theory of large deviations. In terms of two different formulations of time reversal for a same stochastic process, the meanings of {\em dissipative} and {\em conservative} stationary dynamics are discussed.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

A Hamiltonian-Entropy Production Connection in the Skew-symmetric Part of a Stochastic Dynamics

The infinitesimal transition probability operator for a continuous-time discrete-state Markov process, $\mathcal{Q}$, can be decomposed into a symmetric and a skew-symmetric parts. As recently shown for the case of diffusion processes, while the symmetric part corresponding to a gradient system stands for a reversible Markov process, the skew-symmetric part, $\frac{d}{dt}u(t)=\mcA u$, is mathematically equivalent to a linear Hamiltonian dynamics with Hamiltonian $H=1/2u^T\big(\mcA^T\mcA)^{1/2}u$. It can also be transformed into a Schrödinger-like equation $\frac{d}{dt}u=i\mathcal{H}u$ where the "Hamiltonian" operator $\mathcal{H}=-i\mcA$ is Hermitian. In fact, these two representations of a skew-symmetric dynamics emerge natually through singular-value and eigen-value decompositions, respectively. The stationary probability of the Markov process can be expressed as $\|u^s_i\|^2$. The motion can be viewed as "harmonic" since $\frac{d}{dt}\|u(t)-\vec{c}\|^2=0$ where $\vec{c}=(c,c,...,c)$ with $c$ being a constant. More interestingly, we discover that $\textrm{Tr}(\mcA^T\mcA)=\sum_{j,\ell=1}^n \frac{(q_{j\ell}π_\ell-q_{\ell j}π_j)^2}{π_jπ_{\ell}}$, whose right-hand-side is intimately related to the entropy production rate of the Markov process in a nonequilibrium steady state with stationary distribution $\{π_j\}$. The physical implication of this intriguing connection between conservative Hamiltonian dynamics and dissipative entropy production remains to be further explored.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Dissipation, Generalized Free Energy, and a Self-consistent Nonequilibrium Thermodynamics of Chemically Driven Open Subsystems

Nonequilibrium thermodynamics of a system situated in a sustained environment with influx and efflux is usually treated as a subsystem in a larger, closed "universe". It remains a question what the minimally required description for the surrounding of such an open driven system is, so that its nonequilibrium thermodynamics can be established solely based on the internal stochastic kinetics. We provide a solution to this problem using insights from studies of molecular motors in a chemical nonequilibrium steady state (NESS) with sustained external drive through a regenerating system, or in a quasi-steady state (QSS) with an excess amount of ATP, ADP, and Pi. We introduce the key notion of {\em minimal work} that is needed, $W_{min}$, for the external regenerating system to sustain a NESS ({\em e.g.}, maintaining constant concentrations of ATP, ADP and Pi for a molecular motor). Using a Markov (master-equation) description of a motor protein, we illustrate that the NESS and QSS have identical kinetics as well as the Second Law in terms of a same positive entropy production rate. The difference between the heat dissipation of a NESS and its corresponding QSS is exactly the $W_{min}$. This provides a justification for introducing an {\em ideal external regenerating system} and yields a {\em free energy balance equation} between the net free energy input $F_{in}$ and total dissipation $F_{dis}$ in an NESS: $F_{in}$ consists of chemical input minus mechanical output; $F_{dis}$ consists of dissipative heat; and the amount of useful energy becoming heat is the NESS entropy production. Furthermore, we show that for non-stationary systems, the $F_{dis}$ and $F_{in}$ correspond to the entropy production rate and housekeeping heat in stochastic thermodynamics, and identify a relative entropy $H$ as a generalized free energy.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Nonlinear Biochemical Signal Processing via Noise Propagation

Single-cell studies often show significant phenotypic variability due to the stochastic nature of intra-cellular biochemical reactions. When the numbers of molecules, e.g., transcription factors and regulatory enzymes, are in low abundance, fluctuations in biochemical activities become significant and such "noise" can propagate through regulatory cascades in terms of biochemical reaction networks. Here we develop an intuitive, yet fully quantitative method for analyzing how noise affects cellular phenotypes based on identifying a system's nonlinearities and noise propagations. We observe that such noise can simultaneously enhance sensitivities in one behavioral region while reducing sensitivities in another. Employing this novel phenomenon we designed three biochemical signal processing modules: (a) A gene regulatory network that acts as a concentration detector with both enhanced amplitude and sensitivity. (b) A non-cooperative positive feedback system, with a graded dose-response in the deterministic case, that serves as a bistable switch due to noise-induced bimodality. (c) A noise-induced linear amplifier for gene regulation that requires no feedback. The methods developed in the present work allow one to understand and engineer nonlinear biochemical signal processors based on fluctuation-induced phenotypes.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

The Zeroth Law of Thermodynamics and Volume-Preserving Conservative Dynamics with Equilibrium Stochastic Damping

We propose a mathematical formulation of the zeroth law of thermodynamics and develop a stochastic dynamical theory, with a consistent irreversible thermodynamics, for systems possessing sustained conservative stationary current in phase space while in equilibrium with a heat bath. The theory generalizes underdamped mechanical equilibrium: $dx=gdt+\{-D\nablaϕdt+\sqrt{2D}dB(t)\}$, with $\nabla\cdot g=0$ and $\{\cdots\}$ respectively representing phase-volume preserving dynamics and stochastic damping. The zeroth law implies stationary distribution $u^{ss}(x)=e^{-ϕ(x)}$. We find an orthogonality $\nablaϕ\cdot g=0$ as a hallmark of the system. Stochastic thermodynamics based on time reversal $\big(t,ϕ,g\big)\rightarrow\big(-t,ϕ,-g\big)$ is formulated: entropy production $e_p^{\#}(t)=-dF(t)/dt$; generalized "heat" $h_d^{\#}(t)=-dU(t)/dt$, $U(t)=\int_{\mathbb{R}^n} ϕ(x)u(x,t)dx$ being "internal energy", and "free energy" $F(t)=U(t)+\int_{\mathbb{R}^n} u(x,t)\ln u(x,t)dx$ never increases. Entropy follows $\frac{dS}{dt}=e_p^{\#}-h_d^{\#}$. Our formulation is shown to be consistent with an earlier theory of P. Ao. Its contradistinctions to other theories, potential-flux decomposition, stochastic Hamiltonian system with even and odd variables, Klein-Kramers equation, Freidlin-Wentzell's theory, and GENERIC, are discussed.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Analytical Mechanics in Stochastic Dynamics: Most Probable Path, Large-Deviation Rate Function and Hamilton-Jacobi Equation

Analytical (rational) mechanics is the mathematical structure of Newtonian deterministic dynamics developed by D'Alembert, Langrange, Hamilton, Jacobi, and many other luminaries of applied mathematics. Diffusion as a stochastic process of an overdamped individual particle immersed in a fluid, initiated by Einstein, Smoluchowski, Langevin and Wiener, has no momentum since its path is nowhere differentiable. In this exposition, we illustrate how analytical mechanics arises in stochastic dynamics from a randomly perturbed ordinary differential equation $dX_t=b(X_t)dt+εdW_t$ where $W_t$ is a Brownian motion. In the limit of vanishingly small $ε$, the solution to the stochastic differential equation other than $\dot{x}=b(x)$ are all rare events. However, conditioned on an occurence of such an event, the most probable trajectory of the stochastic motion is the solution to Lagrangian mechanics with $\mathcal{L}=\|\dot{q}-b(q)\|^2/4$ and Hamiltonian equations with $H(p,q)=\|p\|^2+b(q)\cdot p$. Hamiltonian conservation law implies that the most probable trajectory for a "rare" event has a uniform "excess kinetic energy" along its path. Rare events can also be characterized by the principle of large deviations which expresses the probability density function for $X_t$ as $f(x,t)=e^{-u(x,t)/ε}$, where $u(x,t)$ is called a large-deviation rate function which satisfies the corresponding Hamilton-Jacobi equation. An irreversible diffusion process with $\nabla\times b\neq 0$ corresponds to a Newtonian system with a Lorentz force $\ddot{q}=(\nabla\times b)\times \dot{q}+1/2\nabla\|b\|^2$. The connection between stochastic motion and analytical mechanics can be explored in terms of various techniques of applied mathematics, for example, singular perturbations, viscosity solutions, and integrable systems.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Landscapes of Non-gradient Dynamics Without Detailed Balance: Stable Limit Cycles and Multiple Attractors

Landscape is one of the key notions in literature on biological processes and physics of complex systems with both deterministic and stochastic dynamics. The large deviation theory (LDT) provides a possible mathematical basis for the scientists' intuition. In terms of Freidlin-Wentzell's LDT, we discuss explicitly two issues in singularly perturbed stationary diffusion processes arisen from nonlinear differential equations: (1) For a process whose corresponding ordinary differential equation has a stable limit cycle, the stationary solution exhibits a clear separation of time scales: an exponential terms and an algebraic prefactor. The large deviation rate function attains its minimum zero on the entire stable limit cycle, while the leading term of the prefactor is inversely proportional to the velocity of the non-uniform periodic oscillation on the cycle. (2) For dynamics with multiple stable fixed points and saddles, there is in general a breakdown of detailed balance among the corresponding attractors. Two landscapes, a local and a global, arise in LDT, and a Markov jumping process with cycle flux emerges in the low-noise limit. A local landscape is pertinent to the transition rates between neighboring stable fixed points; and the global landscape defines a nonequilibrium steady state. There would be nondifferentiable points in the latter for a stationary dynamics with cycle flux. LDT serving as the mathematical foundation for emergent landscapes deserves further investigations.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Mesoscopic Biochemical Basis of Isogenetic Inheritance and Canalization: Stochasticity, Nonlinearity, and Emergent Landscape

Biochemical reaction systems in mesoscopic volume, under sustained environmental chemical gradient(s), can have multiple stochastic attractors. Two distinct mechanisms are known for their origins: ($a$) Stochastic single-molecule events, such as gene expression, with slow gene on-off dynamics; and ($b$) nonlinear networks with feedbacks. These two mechanisms yield different volume dependence for the sojourn time of an attractor. As in the classic Arrhenius theory for temperature dependent transition rates, a landscape perspective provides a natural framework for the system's behavior. However, due to the nonequilibrium nature of the open chemical systems, the landscape, and the attractors it represents, are all themselves {\em emergent properties} of complex, mesoscopic dynamics. In terms of the landscape, we show a generalization of Kramers' approach is possible to provide a rate theory. The emergence of attractors is a form of self-organization in the mesoscopic system; stochastic attractors in biochemical systems such as gene regulation and cellular signaling are naturally inheritable via cell division. Delbrück-Gillespie's mesoscopic reaction system theory, therefore, provides a biochemical basis for spontaneous isogenetic switching and canalization.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

One-Dimensional Birth-Death Process and Delbrück-Gillespie Theory of Mesoscopic Nonlinear Chemical Reactions

As a mathematical theory for the stochasstic, nonlinear dynamics of individuals within a population, Delbrück-Gillespie process (DGP) $n(t)\in\mathbb{Z}^N$, is a birth-death system with state-dependent rates which contain the system size $V$ as a natural parameter. For large $V$, it is intimately related to an autonomous, nonlinear ordinary differential equation as well as a diffusion process. For nonlinear dynamical systems with multiple attractors, the quasi-stationary and stationary behavior of such a birth-death process can be underestood in terms of a separation of time scales by a $T^*\sim e^{αV}$ $(α>0)$: a relatively fast, intra-basin diffusion for $t\ll T^*$ and a much slower inter-basin Markov jump process for $t\gg T^*$. In the present paper for one-dimensional systems, we study both stationary behavior ($t=\infty$) in terms of invariant distribution $p_n^{ss}(V)$, and finite time dynamics in terms of the mean first passsage time (MFPT) $T_{n_1\rightarrow n_2}(V)$. We obtain an asymptotic expression of MFPT in terms of the "stochastic potential" $Φ(x,V)=-(1/V)\ln p^{ss}_{xV}(V)$. We show in general no continuous diffusion process can provide asymptotically accurate representations for both the MFPT and the $p_n^{ss}(V)$ for a DGP. When $n_1$ and $n_2$ belong to two different basins of attraction, the MFPT yields the $T^*(V)$ in terms of $Φ(x,V)\approx ϕ_0(x)+(1/V)ϕ_1(x)$. For systems with a saddle-node bifurcation and catastrophe, discontinuous "phase transition" emerges, which can be characterized by $Φ(x,V)$ in the limit of $V\rightarrow\infty$. In terms of time scale separation, the relation between deterministic, local nonlinear bifurcations and stochastic global phase transition is discussed. The one-dimensional theory is a pedagogic first step toward a general theory of DGP.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Stochastic Physics, Complex Systems and Biology

In complex systems, the interplay between nonlinear and stochastic dynamics, e.g., J. Monod's necessity and chance, gives rise to an evolutionary process in Darwinian sense, in terms of discrete jumps among attractors, with punctuated equilibrium, spontaneous random "mutations" and "adaptations". On an evlutionary time scale it produces sustainable diversity among individuals in a homogeneous population rather than convergence as usually predicted by a deterministic dynamics. The emergent discrete states in such a system, i.e., attractors, have natural robustness against both internal and external perturbations. Phenotypic states of a biological cell, a mesoscopic nonlinear stochastic open biochemical system, could be understood through such a perspective.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Stochastic Thermodynamics Across Scales: Emergent Inter-attractoral Discrete Markov Jump Process and Its Underlying Continuous Diffusion

The consistency across scales of a recently developed mathematical thermodynamic structure, between a continuous stochastic nonlinear dynamical system (diffusion process with Langevin or Fokker-Planck equations) and its emergent discrete, inter-attractoral Markov jump process, is investigated. We analyze how the system's thermodynamic state functions, e.g. free energy $F$, entropy $S$, entropy production $e_p$, and free energy dissipation $\dot{F}$, etc., are related when the continuous system is describe with a coarse-grained discrete variable. We show that the thermodynamics derived from the underlying detailed continuous dynamics is exact in the Helmholtz free-energy representation. That is, the system thermodynamic structure is the same as if one only takes a middle-road and starts with the "natural" discrete description, with the corresponding transition rates empirically determined. By "natural", we mean in the thermodynamic limit of large systems in which there is an inherent separation of time scales between inter- and intra-attractoral dynamics. This result generalizes a fundamental idea from chemistry and the theory of Kramers by including thermodynamics: while a mechanical description of a molecule is in terms of continuous bond lengths and angles, chemical reactions are phenomenologically described by the Law of Mass Action with rate constants, and a stochastic thermodynamics.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

The Dynamics of Zeroth-Order Ultrasensitivity: A Critical Phenomenon in Cell Biology

It is well known since the pioneering work of Goldbeter and Koshland [Proc. Natl. Acad. Sci. USA, vol. 78, pp. 6840-6844 (1981)] that cellular phosphorylation- dephosphorylation cycle (PdPC), catalyzed by kinase and phosphatase under saturated condition with zeroth order enzyme kinetics, exhibits ultrasensitivity, sharp transition. We analyse the dynamics aspects of the zeroth order PdPC kinetics and show a critical slowdown akin to the phase transition in condensed matter physics. We demonstrate that an extremely simple, though somewhat mathematically "singular" model is a faithful representation of the ultrasentivity phenomenon. The simplified mathematical model will be valuable, as a component, in developing complex cellular signaling network theory as well as having a pedagogic value.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

An Information Theoretical Analysis of Kinase Activated Phosphorylation Dephosphorylation Cycle

Signal transduction, the information processing mechanism in biological cells, is carried out by a network of biochemical reactions. The dynamics of driven biochemical reactions can be studied in terms of nonequilibrium statistical physics. Such systems may also be studied in terms of Shannon's information theory. We combine these two perspectives in this study of the basic units (modules) of cellular signaling: the phosphorylation dephosphorylation cycle (PdPC) and the guanosine triphosphatase (GTPase). We show that the channel capacity is zero if and only if the free energy expenditure of biochemical system is zero. In fact, a positive correlation between the channel capacity and free energy expenditure is observed. In terms of the information theory, a linear signaling cascade consisting of multiple steps of PdPC can function as a distributed "multistage code". With increasing number of steps in the cascade, the system trades channel capacity with the code complexity. Our analysis shows that while a static code can be molecular structural based; a biochemical communication channel has to have energy expenditure.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Circular Stochastic Fluctuations in SIS Epidemics with Heterogeneous Contacts Among Sub-populations

The conceptual difference between equilibrium and non-equilibrium steady state (NESS) is well established in physics and chemistry. This distinction, however, is not widely appreciated in dynamical descriptions of biological populations in terms of differential equations in which fixed point, steady state, and equilibrium are all synonymous. We study NESS in a stochastic SIS (susceptible-infectious-susceptible) system with heterogeneous individuals in their contact behavior represented in terms of subgroups. In the infinite population limit, the stochastic dynamics yields a system of deterministic evolution equations for population densities; and for very large but finite system a diffusion process is obtained. We report the emergence of a circular dynamics in the diffusion process, with an intrinsic frequency, near the endemic steady state. The endemic steady state is represented by a stable node in the deterministic dynamics; As a NESS phenomenon, the circular motion is caused by the intrinsic heterogeneity within the subgroups, leading to a broken symmetry and time irreversibility.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Fixation, transient landscape and diffusion's dilemma in stochastic evolutionary game dynamics

Agent-based stochastic models for finite populations have recently received much attention in the game theory of evolutionary dynamics. Both the ultimate fixation and the pre-fixation transient behavior are important to a full understanding of the dynamics. In this paper, we study the transient dynamics of the well-mixed Moran process through constructing a landscape function. It is shown that the landscape playing a central theoretical "device" that integrates several lines of inquiries: the stable behavior of the replicator dynamics, the long-time fixation, and continuous diffusion approximation associated with asymptotically large population. Several issues relating to the transient dynamics are discussed: (i) multiple time scales phenomenon associated with intra- and inter-attractoral dynamics; (ii) discontinuous transition in stochastically stationary process akin to Maxwell construction in equilibrium statistical physics; and (iii) the dilemma diffusion approximation facing as a continuous approximation of the discrete evolutionary dynamics. It is found that rare events with exponentially small probabilities, corresponding to the uphill movements and barrier crossing in the landscape with multiple wells that are made possible by strong nonlinear dynamics, plays an important role in understanding the origin of the complexity in evolutionary, nonlinear biological systems.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Hill's Small Systems Nanothermodynamics: A Simple Macromolecular Partition Problem with a Statistical Perspective

Using a simple example of biological macromolecules which are partitioned between bulk solution and membrane, we investigate T.L. Hill's phenomenological nanothermodynamics for small systems. By introducing a {\em systems size dependent} equilibrium constant for the bulk-membrane partition, we obtain Hill's results on differential and integral chemical potentials $μ$ and $\hatμ$ from computations based on standard Gibbsian equilibrium statistical mechanics. It is shown that their difference can be understood from an equilibrium re-partitioning between bulk and membrane fractions upon a change in system's size; it is closely related to the system's fluctuations and inhomogeneity. These results provide a better understanding of the nanothermodynamics and clarify its logical relation with the theory of statistical mechanics.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Irreversible Thermodynamics in Multiscale Stochastic Dynamical Systems

This work extends the results of the recently developed theory of a rather complete thermodynamic formalism for discrete-state, continuous-time Markov processes with and without detailed balance. We aim at investigating the question that whether and how the thermodynamic structure is invariant in a multiscale stochastic system. That is, whether the relations between thermodynamic functions of state and process variables remain unchanged when the system is viewed at different time scales and resolutions. Our results show that the dynamics on a fast time scale contribute an entropic term to the "internal energy function", $u_S(x)$, for the slow dynamics. Based on the conditional free energy $u_S(x)$, one can then treat the slow dynamics as if the fast dynamics is nonexistent. Furthermore, we show that the free energy, which characterizes the spontaneous organization in a system without detailed balance, is invariant with or without the fast dynamics: The fast dynamics is assumed to reach stationarity instantaneously on the slow time scale; they have no effect on the system's free energy. The same can not be said for the entropy and the internal energy, both of which contain the same contribution from the fast dynamics. We also investigate the consequences of time-scale separation in connection to the concepts of quasi-stationaryty and steady-adiabaticity introduced in the phenomenological steady-state thermodynamics.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Maximum Entropy Principle, Equal Probability a Priori and Gibbs Paradox

We prove that information-theoretic maximum entropy (MaxEnt) approach to canonical ensemble is mathematically equivalent to the classic approach of Boltzmann, Gibbs and Darwin-Fowler. The two approaches, however, "interpret" a same mathematical theorem differently; most notably observing mean-energy in the former and energy conservation in the latter. However, applying the same MaxEnt method to grand canonical ensemble fails; while carefully following the classic approach based on Boltzmann's microcanonical {\em equal probability a priori} produces the correct statistics: One does not need to invoke quantum mechanics; and there is no Gibbs paradox. MaxEnt and related minimum relative entropy principle are based on the mathematical theorem concerning large deviations of rare fluctuations. As a scientific method, it requires classic mechanics, or some other assumptions, to provide meaningful {\em prior distributions} for the expected-value based statistical inference. A naive assumption of uniform prior is not valid in statistical mechanics.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Thermodynamics and Geometry of Reversible and Irreversible Markov Processes

Master equation with microscopic reversibility ($q_{ij}\neq 0$ iff $q_{ji}\neq 0$) has a {\em thermodynamic superstructure} in terms of two state functions $S$, entropy, and $F$, free energy: It is discovered recently that entropy production rate $e_p=-dF/dt+Q_{hk}$ with both $-dF/dt=f_d, Q_{hk} \ge 0$. The free energy dissipation $f_d\ge 0$ reflects irreversibility in spontaneous self-organization; house-keeping heat $Q_{hk}\ge 0$ reveals broken time-symmetry in open system driven away from equilibrium. In a Riemannian geometric space, the master equation is a geodesic flow when $Q_{hk}=0$; here we show that the $e_p$ decomposition is orthogonal: $e_p$, $f_d$, $Q_{hk}$ forms a pythagorean triples. Gradient flow means {\em maximum dissipation principle} outside Onsager's regime. The presence of $Q_{hk}$ makses gradient flow no longer generally true. Thermodynamics of stochastic physics requires a new geometric perspective.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

An annotated English translation of `Kinetics of stationary reactions' [M. I. Temkin, Dolk. Akad. Nauk SSSR. 152, 156 (1963)]

Temkin's 1963 article on one-way fluxes and flux ratios in steady-state reaction systems bears directly on current research in physical and biological chemistry, such as in the interpretation of metabolic exchange fluxes determined from isotopomer labeling experiments. Yet, originally published in Russian [Dolk. Akad. Nauk SSSR 152, 156-159 (1963)], this article has remained inaccessible to much of the scientific community. Here we provide an English translation of the original article with several additional clarifications and corrections.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

van't Hoff-Arrhenius Analysis of Mesoscopic and Macroscopic Dynamics of Simple Biochemical Systems: Stochastic vs. Nonlinear Bistabilities

Multistability of mesoscopic, driven biochemical reaction systems has implications to a wide range of cellular processes. Using several simple models, we show that one class of bistable chemical systems has a deterministic counterpart in the nonlinear dynamics based on the Law of Mass Action, while another class, widely known as noise-induced stochastic bistability, does not. Observing the system's volume ($V$) playing a similar role as the inverse temperature ($β$) in classical rate theory, an van't Hoff-Arrhenius like analysis is introduced. In one-dimensional systems, a transition rate between two states, represented in terms of a barrier in the landscape for the dynamics $Φ(x,V)$, $k\propto\exp\{-VΔΦ^‡(V)\}$, can be understood from a decomposition $ΔΦ^‡(V) \approxΔϕ_0^‡ Δϕ_1^‡/V$. Nonlinear bistability means $Δϕ_0^‡>0$ while stochastic bistability has $Δϕ_0^‡<0$ but $Δϕ_1^‡>0$. Stochastic bistabilities can be viewed as remants (or "ghosts) of nonlinear bifurcations or extinction phenomenon, and $Δϕ_0^‡$ and $Δϕ_1^‡$ as "enthalpic" and "entropic" barriers to a transition.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

The Physical Origins of Entropy Production, Free Energy Dissipation and their Mathematical Representations

A complete mathematical theory of nonequilibrium thermodynamics of stochastic systems in terms of master equations is presented. As generalizations of isothermal entropy and free energy, two functions of states play central roles: the Gibbs entropy $S$ and the relative entropy $F$, which are related via the stationary distribution of the stochastic dynamics. $S$ satisfies the fundamental entropy balance equation $dS/dt=e_p-h_d/T$ with entropy production rate $e_p\ge 0$ and heat dissipation rate $h_d$, while $dF/dt=-f_d\le 0$. For closed systems that satisfy detailed balance: $Te_p(t)=f_d(t)$. For open system one has $Te_p(t)=f_d(t)+Q_{hk}(t)$ where the housekeeping heat $Q_{hk}\ge 0$ was first introduced in the phenomenological nonequilibrium steady state thermodynamics. Entropy production $e_p$ consists of free energy dissipation associated with spontaneous relaxation, $f_d$, and active energy pumping that sustains the open system $Q_{hk}$. The amount of excess heat involved in the relaxation $Q_{ex}=h_d-Q_{hk} = f_d-T(dS/dt)$.

0 citations0 reviewsNo code linkedOpen access
preprint2009arXiv

Thermodynamic Limit of a Nonequilibrium Steady-State: Maxwell-Type Construction for a Bistable Biochemical System

We show that the thermodynamic limit of a bistable phosphorylation-dephosphorylation cycle has a selection rule for the "more stable" macroscopic steady state. The analysis is akin to the Maxwell construction. Based on the chemical master equation approach, it is shown that, except at a critical point, bistability disappears in the stochastic model when fluctuation is sufficiently low but unneglectable. Onsager's Gaussian fluctuation theory applies to the unique macroscopic steady state. With initial state in the basin of attraction of the "less stable" steady state, the deterministic dynamics obtained by the Law of Mass Action is a metastable phenomenon. Stability and robustness in cell biology are stochastic concepts.

0 citations0 reviewsNo code linkedOpen access
preprint2005arXiv

Nonequilibrium Potential Function of Chemically Driven Single Macromolecules via Jarzynski-Type Log-Mean-Exponential Work

Applying the method from recently developed fluctuation theorems to the stochastic dynamics of single macromolecules in ambient fluid at constant temperature, we establish two Jarzynski-type equalities: (1) between the log-mean-exponential (LME) of the irreversible heat dissiption of a driven molecule in nonequilibrium steady-state (NESS) and $\ln P^{ness}(x)$, and (2) between the LME of the work done by the internal force of the molecule and nonequilibrium chemical potential function $μ^{ness}(x)$ $\equiv U(x)+k_BT\ln P^{ness}(x)$, where $P^{ness}(x)$ is the NESS probability density in the phase space of the macromolecule and $U(x)$ is its internal potential function. $Ψ$ = $\intμ^{ness}(x)P^{ness}(x)dx$ is shown to be a nonequilibrium generalization of the Helmholtz free energy and $ΔΨ$ = $ΔU-TΔS$ for nonequilibrium processes, where $S$ $=-k_B\int P(x)\ln P(x)dx$ is the Gibbs entropy associated with $P(x)$. LME of heat dissipation generalizes the concept of entropy, and the equalities define thermodynamic potential functions for open systems far from equilibrium.

0 citations0 reviewsNo code linkedOpen access
preprint2001arXiv

A Stochastic Analysis of a Brownian Ratchet Model for Actin-Based Motility and Integrate-and-Firing Neurons

In recent single-particle tracking (SPT) measurements on {\it Listeria monocytogenes} motility {\em in vitro}, the actin-based stochastic dynamics of the bacterium movement is analyzed statistically (Kuo and McGrath, 2000). The mean-square displacement (MSD) of the detrended trajectory exhibit a linear behavior; it has been suggested that a corresponding analysis for the Brownian ratchet model (Peskin, Odell, & Oster, 1993) leads to a non-monotonic MSD. A simplified version of the Brownian ratchet, when its motion is limited by the bacterium movement, is proposed and analyzed stochastically. Analytical results for the simple model are obtained and statistical data analysis is investigated. The MSD of the stochastic bacterium movement is a quadratic function while the MSD for the detrended trajectory is shown to be linear. The mean velocity and effective diffusion constant of the propelled bacterium in the long-time limit, and the short-time relaxation are obtained from the MSD analysis. The MSD of the gap between actin and the bacterium exhibits an oscillatory behavior when there is a large resistant force from the bacterium. The stochastic model for actin-based motility is also mathematically equivalent to a model for integrate-and-firing neurons. Hence our mathematical results have applications in other biological problems. For comparison, a continuous formalism of the BR model with great analytical simplicity is also studied.

0 citations0 reviewsNo code linkedOpen access