BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Peter W. Glynn

Peter W. Glynn appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
math.PRmath.NANumerical AnalysisComputational Engineering, Finance, and ScienceMachine Learningmath.OCmath.STq-fin.CPStatistics TheoryComputationInformation Theorymath.ITmath.SPMethodologyQuantitative MethodsSystems and Control

Catalog footprint

What is connected

15works
16topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

15 published item(s)

preprint2022arXiv

A New Truncation Algorithm for Markov Chain Equilibrium Distributions with Computable Error Bounds

This paper introduces a new algorithm for numerically computing equilibrium (i.e. stationary) distributions for Markov chains and Markov jump processes with either a very large finite state space or a countably infinite state space. The algorithm is based on a ratio representation for equilibrium expectations in which the numerator and denominator correspond to expectations defined over paths that start and end within a given return set $K$. When $K$ is a singleton, this representation is a well-known consequence of regenerative process theory. For computational tractability, we ignore contributions to the path expectations corresponding to excursions out of a given truncation set $A$. This yields a truncation algorithm that is provably convergent as $A$ gets large. Furthermore, in the presence of a suitable Lyapunov function, we can bound the path expectations, thereby providing computable and convergent error bounds for our numerical procedure. Our paper also provides a computational comparison with two other truncation methods that come with computable error bounds. The results are in alignment with the observation that our bounds have associated computational complexities that typically scale better as the truncation set gets bigger.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

A Short Proof of a Convex Representation for Stationary Distributions of Markov Chains with an Application to State Space Truncation

In an influential paper, Courtois and Semal (1984) establish that when $G$ is an irreducible substochastic matrix for which $\sum_{n=0}^{\infty}G^n <\infty$, then the stationary distribution of any stochastic matrix $P\ge G$ can be expressed as a convex combination of the normalized rows of $(I-G)^{-1} = \sum_{n=0}^{\infty} G^n$. In this note, we give a short proof of this result that extends the theory to the countably infinite and continuous state space settings. This result plays an important role in obtaining error bounds in algorithms involving nearly decomposable Markov chains, and also in state truncations for Markov chains. We also use the representation to establish a new total variation distance error bound for truncated Markov chains.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

On Convergence of a Truncation Scheme for Approximating Stationary Distributions of Continuous State Space Markov Chains and Processes

In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior of the truncated chain or process is close to that of the untruncated system. For example, such questions arise naturally when considering numerical methods for computing stationary distributions on unbounded state space. In this paper, we use the principle of "regeneration" to show that the stationary distributions of "fixed state" truncations converge in great generality (in total variation norm) to the stationary distribution of the untruncated limit, when the untruncated chain is positive Harris recurrent. Even in countable state space, our theory extends known results by showing that the augmentation can correspond to an $r$-regular measure. In addition, we extend our theory to cover an important subclass of Harris recurrent Markov processes that include non-explosive Markov jump processes on countable state space.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

On Convergence of General Truncation-Augmentation Schemes for Approximating Stationary Distributions of Markov Chains

In the analysis of Markov chains and processes, it is sometimes convenient to replace an unbounded state space with a "truncated" bounded state space. When such a replacement is made, one often wants to know whether the equilibrium behavior of the truncated chain or process is close to that of the untruncated system. For example, such questions arise naturally when considering numerical methods for computing stationary distributions on unbounded state space. In this paper, we study general truncation-augmentation schemes, in which the substochastic truncated "northwest corner" of the transition matrix or kernel is stochasticized (or augmented) arbitrarily. In the presence of a Lyapunov condition involving a coercive function, we show that such schemes are generally convergent in countable state space, provided that the truncation is chosen as a sublevel set of the Lyapunov function. For stochastically monotone Markov chains on $\mathbb Z_+$, we prove that we can always choose the truncation sets to be of the form $\{0,1,...,n\}$. We then provide sufficient conditions for weakly continuous Markov chains under which general truncation-augmentation schemes converge weakly in continuous state space. Finally, we briefly discuss the extension of the theory to continuous time Markov jump processes.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Multivariate Distributionally Robust Convex Regression under Absolute Error Loss

This paper proposes a novel non-parametric multidimensional convex regression estimator which is designed to be robust to adversarial perturbations in the empirical measure. We minimize over convex functions the maximum (over Wasserstein perturbations of the empirical measure) of the absolute regression errors. The inner maximization is solved in closed form resulting in a regularization penalty involves the norm of the gradient. We show consistency of our estimator and a rate of convergence of order $ \widetilde{O}\left( n^{-1/d}\right) $, matching the bounds of alternative estimators based on square-loss minimization. Contrary to all of the existing results, our convergence rates hold without imposing compactness on the underlying domain and with no a priori bounds on the underlying convex function or its gradient norm.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Sequential Batch Learning in Finite-Action Linear Contextual Bandits

We study the sequential batch learning problem in linear contextual bandits with finite action sets, where the decision maker is constrained to split incoming individuals into (at most) a fixed number of batches and can only observe outcomes for the individuals within a batch at the batch's end. Compared to both standard online contextual bandits learning or offline policy learning in contexutal bandits, this sequential batch learning problem provides a finer-grained formulation of many personalized sequential decision making problems in practical applications, including medical treatment in clinical trials, product recommendation in e-commerce and adaptive experiment design in crowdsourcing. We study two settings of the problem: one where the contexts are arbitrarily generated and the other where the contexts are \textit{iid} drawn from some distribution. In each setting, we establish a regret lower bound and provide an algorithm, whose regret upper bound nearly matches the lower bound. As an important insight revealed therefrom, in the former setting, we show that the number of batches required to achieve the fully online performance is polynomial in the time horizon, while for the latter setting, a pure-exploitation algorithm with a judicious batch partition scheme achieves the fully online performance even when the number of batches is less than logarithmic in the time horizon. Together, our results provide a near-complete characterization of sequential decision making in linear contextual bandits when batch constraints are present.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

A Generalized Fundamental Matrix for Computing Fundamental Quantities of Markov Systems

As is well known, the fundamental matrix $(I - P + e π)^{-1}$ plays an important role in the performance analysis of Markov systems, where $P$ is the transition probability matrix, $e$ is the column vector of ones, and $π$ is the row vector of the steady state distribution. It is used to compute the performance potential (relative value function) of Markov decision processes under the average criterion, such as $g=(I - P + e π)^{-1} f$ where $g$ is the column vector of performance potentials and $f$ is the column vector of reward functions. However, we need to pre-compute $π$ before we can compute $(I - P + e π)^{-1}$. In this paper, we derive a generalization version of the fundamental matrix as $(I - P + e r)^{-1}$, where $r$ can be any given row vector satisfying $r e \neq 0$. With this generalized fundamental matrix, we can compute $g=(I - P + e r)^{-1} f$. The steady state distribution is computed as $π= r(I - P + e r)^{-1}$. The Q-factors at every state-action pair can also be computed in a similar way. These formulas may give some insights on further understanding how to efficiently compute or estimate the values of $g$, $π$, and Q-factors in Markov systems, which are fundamental quantities for the performance optimization of Markov systems.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Measuring the Initial Transient: Reflected Brownian Motion

We analyze the convergence to equilibrium of one-dimensional reflected Brownian motion (RBM) and compute a number of related initial transient formulae. These formulae are of interest as approximations to the initial transient for queueing systems in heavy traffic, and help us to identify settings in which initialization bias is significant. We conclude with a discussion of mean square error for RBM. Our analysis supports the view that initial transient effects for RBM and related models are typically of modest size relative to the intrinsic stochastic variability, unless one chooses an especially poor initialization.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Central Limit Theorems and Large Deviations for Additive Functionals of Reflecting Diffusion Processes

This paper develops central limit theorems (CLT's) and large deviations results for additive functionals associated with reflecting diffusions in which the functional may include a term associated with the cumulative amount of boundary reflection that has occurred. Extending the known central limit and large deviations theory for Markov processes to include additive functionals that incorporate boundary reflection is important in many applications settings in which reflecting diffusions arise, including queueing theory and economics. In particular, the paper establishes the partial differential equations that must be solved in order to explicitly compute the mean and variance for the CLT, as well as the associated rate function for the large deviations principle.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Exact Estimation for Markov Chain Equilibrium Expectations

We introduce a new class of Monte Carlo methods, which we call exact estimation algorithms. Such algorithms provide unbiased estimators for equilibrium expectations associated with real- valued functionals defined on a Markov chain. We provide easily implemented algorithms for the class of positive Harris recurrent Markov chains, and for chains that are contracting on average. We further argue that exact estimation in the Markov chain setting provides a significant theoretical relaxation relative to exact simulation methods.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Exact Simulation of Non-stationary Reflected Brownian Motion

This paper develops the first method for the exact simulation of reflected Brownian motion (RBM) with non-stationary drift and infinitesimal variance. The running time of generating exact samples of non-stationary RBM at any time $t$ is uniformly bounded by $\mathcal{O}(1/\barγ^2)$ where $\barγ$ is the average drift of the process. The method can be used as a guide for planning simulations of complex queueing systems with non-stationary arrival rates and/or service time.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Shape-constrained Estimation of Value Functions

We present a fully nonparametric method to estimate the value function, via simulation, in the context of expected infinite-horizon discounted rewards for Markov chains. Estimating such value functions plays an important role in approximate dynamic programming and applied probability in general. We incorporate "soft information" into the estimation algorithm, such as knowledge of convexity, monotonicity, or Lipchitz constants. In the presence of such information, a nonparametric estimator for the value function can be computed that is provably consistent as the simulated time horizon tends to infinity. As an application, we implement our method on price tolling agreement contracts in energy markets.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

A new approach to unbiased estimation for SDE's

In this paper, we introduce a new approach to constructing unbiased estimators when computing expectations of path functionals associated with stochastic differential equations (SDEs). Our randomization idea is closely related to multi-level Monte Carlo and provides a simple mechanism for constructing a finite variance unbiased estimator with "square root convergence rate" whenever one has available a scheme that produces strong error of order greater than 1/2 for the path functional under consideration.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

On the Convergence of Finite Order Approximations of Stationary Time Series

The approximation of a stationary time-series by finite order autoregressive (AR) and moving averages (MA) is a problem that occurs in many applications. In this paper we study asymptotic behavior of the spectral density of finite order approximations of wide sense stationary time series. It is shown that when the on the spectral density is non-vanishing in $[-π,π]$ and the covariance is summable, the spectral density of the approximating autoregressive sequence converges at the origin. Under additional mild conditions on the coefficients of the Wold decomposition it is also shown that the spectral densities of both moving average and autoregressive approximations converge in $L_2$ as the order of approximation increases.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Uniform Approximations for the M/G/1 Queue with Subexponential Processing Times

This paper studies the asymptotic behavior of the steady-state waiting time, W_infty, of the M/G/1 queue with subexponenential processing times for different combinations of traffic intensities and overflow levels. In particular, we provide insights into the regions of large deviations where the so-called heavy traffic approximation and heavy tail asymptotic hold. For queues whose service time distribution decays slower than e^{-sqrt{t}} we identify a third region of asymptotics where neither the heavy traffic nor the heavy tailed approximations are valid. These results are obtained by deriving approximations for P(W_infty > x) that are either uniform in the traffic intensity as the tail value goes to infinity or uniform on the positive axis as the traffic intensity converges to one. Our approach makes clear the connection between the asymptotic behavior of the steady-state waiting time distribution and that of an associated random walk.

0 citations0 reviewsNo code linkedOpen access