Source author record

Kai Shao

Kai Shao appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Catalog footprint

What is connected

3works
8topics
4close collaborators

Actions

Connect this record

Log in to claim

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

3 published item(s)

preprint2022arXiv

Cyclotron quantization and mirror-time transition on nonreciprocal lattices

Unidirectional transport and localized cyclotron motion are two opposite physical phenomena. Here, we study the interplay effects between them on nonreciprocal lattices subject to a magnetic field. We show that, in the long-wavelength limit, the trajectories of the wave packets always form closed orbits in four-dimensional (4D) complex space. Therefore, the semiclassical quantization rules persist despite the nonreciprocity, which preserves real Landau levels. We predict a different type of non-Hermitian spectral transition induced by the spontaneous breaking of the combined mirror-time reversal ($\mathcal{MT}$) symmetry, which generally exists in such systems. An order parameter is proposed to describe the $\mathcal{MT}$ phase transition, not only to determine the $\mathcal{MT}$ phase boundary but also to quantify the degree of $\mathcal{MT}$-symmetry breaking. Such an order parameter can be generally applied to all types of non-Hermitian phase transitions.

preprint2022arXiv

On the Convergence of the Monte Carlo Exploring Starts Algorithm for Reinforcement Learning

A simple and natural algorithm for reinforcement learning (RL) is Monte Carlo Exploring Starts (MCES), where the Q-function is estimated by averaging the Monte Carlo returns, and the policy is improved by choosing actions that maximize the current estimate of the Q-function. Exploration is performed by "exploring starts", that is, each episode begins with a randomly chosen state and action, and then follows the current policy to the terminal state. In the classic book on RL by Sutton & Barto (2018), it is stated that establishing convergence for the MCES algorithm is one of the most important remaining open theoretical problems in RL. However, the convergence question for MCES turns out to be quite nuanced. Bertsekas & Tsitsiklis (1996) provide a counter-example showing that the MCES algorithm does not necessarily converge. Tsitsiklis (2002) further shows that if the original MCES algorithm is modified so that the Q-function estimates are updated at the same rate for all state-action pairs, and the discount factor is strictly less than one, then the MCES algorithm converges. In this paper we make headway with the original and more efficient MCES algorithm given in Sutton & Barto (1998), establishing almost sure convergence for Optimal Policy Feed-Forward MDPs, which are MDPs whose states are not revisited within any episode when using an optimal policy. Such MDPs include a large class of environments such as all deterministic environments and all episodic environments with a timestep or any monotonically changing values as part of the state. Different from the previous proofs using stochastic approximations, we introduce a novel inductive approach, which is very simple and only makes use of the strong law of large numbers.

preprint2022arXiv

Vortices and cosmic strings in a generalized Born-Infeld model

In this paper, we consider two types of topological solitons in a generalized Born-Infeld-Higgs model. We explore the self-dual structure of the model and prove the existence of planar vortex solutions. Furthermore, we couple the system with the Einstein equations and study the cosmic strings problem over $\mathbb R^{1,1}\times S$, where $S$ is a Riemann surface. We prove the existence of cosmic string solutions when $S$ is noncompact. We also discuss the decay estimates for vortices and cosmic strings at infinity and show that the minimal energy is quantized and depends on the number of vortices and strings, respectively.