BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

J. G. Dai

J. G. Dai appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
math.PRMachine Learningmath.OC

Catalog footprint

What is connected

12works
3topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

12 published item(s)

preprint2022arXiv

High order steady-state diffusion approximations

We derive and analyze new diffusion approximations of stationary distributions of Markov chains that are based on second- and higher-order terms in the expansion of the Markov chain generator. Our approximations achieve a higher degree of accuracy compared to diffusion approximations widely used for the past fifty years, while retaining a similar computational complexity. To support our approximations, we present a combination of theoretical and numerical results across three different models. Our approximations are derived recursively through Stein/Poisson equations, and the theoretical results are proved using Stein's method.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Queueing Network Controls via Deep Reinforcement Learning

Novel advanced policy gradient (APG) methods, such as Trust Region policy optimization and Proximal policy optimization (PPO), have become the dominant reinforcement learning algorithms because of their ease of implementation and good practical performance. A conventional setup for notoriously difficult queueing network control problems is a Markov decision problem (MDP) that has three features: infinite state space, unbounded costs, and long-run average cost objective. We extend the theoretical framework of these APG methods for such MDP problems. The resulting PPO algorithm is tested on a parallel-server system and large-size multiclass queueing networks. The algorithm consistently generates control policies that outperform state-of-art heuristics in literature in a variety of load conditions from light to heavy traffic. These policies are demonstrated to be near-optimal when the optimal policy can be computed. A key to the successes of our PPO algorithm is the use of three variance reduction techniques in estimating the relative value function via sampling. First, we use a discounted relative value function as an approximation of the relative value function. Second, we propose regenerative simulation to estimate the discounted relative value function. Finally, we incorporate the approximating martingale-process method into the regenerative estimator.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

High order steady-state diffusion approximation of the Erlang-C system

In this paper we introduce a new diffusion approximation for the steady-state customer count of the Erlang-C system. Unlike previous diffusion approximations, which use the steady-state distribution of a diffusion process with a constant diffusion coefficient, our approximation uses the steady-state distribution of a diffusion process with a \textit{state-dependent} diffusion coefficient. We show, both analytically and numerically, that our new approximation is an order of magnitude better than its counterpart. To obtain the analytical results, we use Stein's to show that a variant of the Wasserstein distance between the normalized customer count distribution and our approximation vanishes at a rate of $1/R$, where $R$ is the offered load to the system. In contrast, the previous approximation only achieved a rate of $1/R$. We hope our results motivate others to consider diffusion approximations with state-dependent diffusion coefficients.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Stein's method for steady-state diffusion approximations of $M/Ph/n+M$ systems

We consider $M/Ph/n+M$ queueing systems in steady state. We prove that the Wasserstein distance between the stationary distribution of the normalized system size process and that of a piecewise Ornstein-Uhlenbeck (OU) process is bounded by $C/\sqrtλ$, where the constant $C$ is independent of the arrival rate $λ$ and the number of servers $n$ as long as they are in the Halfin-Whitt parameter regime. For each integer $m>0$, we also establish a similar bound for the difference of the $m$th steady-state moments. For the proofs, we develop a modular framework that is based on Stein's method. The framework has three components: Poisson equation, generator coupling, and state space collapse. The framework, with further refinement, is likely applicable to steady-state diffusion approximations for other stochastic systems.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Technical Note for Discrete-Time Diffusion Approximations Motivated from Hospital Inpatient Flow Management

This note details the development of a discrete-time diffusion process to approximate the midnight customer count process in a $M_\textrm{per}/\textrm{Geo}_\textrm{2timeScale}/N$ system. We prove a limit theorem that supports this diffusion approximation, and discuss two methods to compute the stationary distribution of this discrete-time diffusion process.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

A multi-dimensional SRBM: Geometric views of its product form stationary distribution

We present a geometric interpretation of a product form stationary distribution for a $d$-dimensional semimartingale reflecting Brownian motion (SRBM) that lives in the nonnegative orthant. The $d$-dimensional SRBM data can be equivalently specified by $d+1$ geometric objects: an ellipse and $d$ rays. Using these geometric objects, we establish necessary and sufficient conditions for characterizing product form stationary distribution. The key idea in the characterization is that we decompose the $d$-dimensional problem to $\frac{1}{2}d(d-1)$ two-dimensional SRBMs, each of which is determined by an ellipse and two rays. This characterization contrasts with the algebraic condition of [14]. A $d$-station tandem queue example is presented to illustrate how the product form can be obtained using our characterization. Drawing the two-dimensional results in [1,7], we discuss potential optimal paths for a variational problem associated with the three-station tandem queue. Except Appendix D, the rest of this paper is almost identical to the QUESTA paper with the same title.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Decomposable stationary distribution of a multidimensional SRBM

We call a multidimensional distribution to be decomposable with respect to a partition of two sets of coordinates if the original distribution is the product of the marginal distributions associated with these two sets. We focus on the stationary distribution of a multidimensional semimartingale reflecting Brownian motion (SRBM) on a nonnegative orthant. An SRBM is uniquely determined (in distribution) by its data that consists of a covariance matrix, a drift vector, and a reflection matrix. Assume that the stationary distribution of an SRBM exists. We first characterize two marginal distributions under the decomposability assumption. We prove that they are the stationary distributions of some lower dimensional SRBMs. We also identify the data for these lower dimensional SRBMs. Thus, under the decomposability assumption, we can obtain the stationary distribution of the original SRBM by computing those of the lower dimensional ones. However, this characterization of the marginal distributions is not sufficient for the decomposability. So, we next consider necessary and sufficient conditions for the decomposability. We obtain those conditions for several classes of SRBMs. These classes include SRBMs arising from Brownian models of queueing networks that have two sets of stations with feed-forward routing between these two sets. This work is motivated by applications of SRBMs and geometric interpretations of the product form stationary distributions.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Validity of heavy-traffic steady-state approximations in many-server queues with abandonment

We consider GI/Ph/n+M parallel-server systems with a renewal arrival process, a phase-type service time distribution, n homogenous servers, and an exponential patience time distribution with positive rate. We show that in the Halfin-Whitt regime, the sequence of stationary distributions corresponding to the normalized state processes is tight. As a consequence, we establish an interchange of heavy traffic and steady state limits for GI/Ph/n+M queues.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Diffusion limits of limited processor sharing queues

We consider a processor sharing queue where the number of jobs served at any time is limited to $K$, with the excess jobs waiting in a buffer. We use random counting measures on the positive axis to model this system. The limit of this measure-valued process is obtained under diffusion scaling and heavy traffic conditions. As a consequence, the limit of the system size process is proved to be a piece-wise reflected Brownian motion.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Many-server queues with customer abandonment: numerical analysis of their diffusion models

We use multidimensional diffusion processes to approximate the dynamics of a queue served by many parallel servers. The queue is served in the first-in-first-out (FIFO) order and the customers waiting in queue may abandon the system without service. Two diffusion models are proposed in this paper. They differ in how the patience time distribution is built into them. The first diffusion model uses the patience time density at zero and the second one uses the entire patience time distribution. To analyze these diffusion models, we develop a numerical algorithm for computing the stationary distribution of such a diffusion process. A crucial part of the algorithm is to choose an appropriate reference density. Using a conjecture on the tail behavior of a limit queue length process, we propose a systematic approach to constructing a reference density. With the proposed reference density, the algorithm is shown to converge quickly in numerical experiments. These experiments also show that the diffusion models are good approximations for many-server queues, sometimes for queues with as few as twenty servers.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Many-server diffusion limits for $G/Ph/n+GI$ queues

This paper studies many-server limits for multi-server queues that have a phase-type service time distribution and allow for customer abandonment. The first set of limit theorems is for critically loaded $G/Ph/n+GI$ queues, where the patience times are independent and identically distributed following a general distribution. The next limit theorem is for overloaded $G/ Ph/n+M$ queues, where the patience time distribution is restricted to be exponential. We prove that a pair of diffusion-scaled total-customer-count and server-allocation processes, properly centered, converges in distribution to a continuous Markov process as the number of servers $n$ goes to infinity. In the overloaded case, the limit is a multi-dimensional diffusion process, and in the critically loaded case, the limit is a simple transformation of a diffusion process. When the queues are critically loaded, our diffusion limit generalizes the result by Puhalskii and Reiman (2000) for $GI/Ph/n$ queues without customer abandonment. When the queues are overloaded, the diffusion limit provides a refinement to a fluid limit and it generalizes a result by Whitt (2004) for $M/M/n/+M$ queues with an exponential service time distribution. The proof techniques employed in this paper are innovative. First, a perturbed system is shown to be equivalent to the original system. Next, two maps are employed in both fluid and diffusion scalings. These maps allow one to prove the limit theorems by applying the standard continuous-mapping theorem and the standard random-time-change theorem.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Positive recurrence of reflecting Brownian motion in three dimensions

Consider a semimartingale reflecting Brownian motion (SRBM) $Z$ whose state space is the $d$-dimensional nonnegative orthant. The data for such a process are a drift vector $θ$, a nonsingular $d\times d$ covariance matrix $Σ$, and a $d\times d$ reflection matrix $R$ that specifies the boundary behavior of $Z$. We say that $Z$ is positive recurrent, or stable, if the expected time to hit an arbitrary open neighborhood of the origin is finite for every starting state. In dimension $d=2$, necessary and sufficient conditions for stability are known, but fundamentally new phenomena arise in higher dimensions. Building on prior work by El Kharroubi, Ben Tahar and Yaacoubi [Stochastics Stochastics Rep. 68 (2000) 229--253, Math. Methods Oper. Res. 56 (2002) 243--258], we provide necessary and sufficient conditions for stability of SRBMs in three dimensions; to verify or refute these conditions is a simple computational task. As a byproduct, we find that the fluid-based criterion of Dupuis and Williams [Ann. Probab. 22 (1994) 680--702] is not only sufficient but also necessary for stability of SRBMs in three dimensions. That is, an SRBM in three dimensions is positive recurrent if and only if every path of the associated fluid model is attracted to the origin. The problem of recurrence classification for SRBMs in four and higher dimensions remains open.

0 citations0 reviewsNo code linkedOpen access