BZPEERReading, trust and collaboration for research
Menu

Discover

GraphFollow connections around a paper, topic or saved view.

Workspaces

HomePick up where your research flow left off.InboxHandle requests, replies and notifications from one place.CollectionsCurate reading lists, evidence sets and shareable dossiers.ResearchSee your private provenance graph, trail and imports.CollaborationMove from discovery into focused discussion rooms.PublishDraft, preview and publish full-text research articles.

Network

ExpertsFind specialists worth following or asking for help.CommunitiesJoin research circles organized around shared work.PartnersSee external organizations active around the same topics.

Opportunities

ListingsBrowse roles, calls and collaborations with clearer fit signals.

Account

Log inReturn to your reading and collaboration workspace.Create accountStart saving work, following people and joining teams.
Log inCreate account

Researcher profile

Jun Luo

Jun Luo contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate
Artificial IntelligenceMachine LearningComputer Visiongr-qcRoboticsastro-ph.IMmath.GNphysics.ins-detcond-mat.mtrl-scicond-mat.supr-conCryptography and Securityeess.ASeess.SPmath.MGmath.OCMethodologyMultiagent Systemsphysics.atom-phSocial and Information NetworksSound

Trust snapshot

Quick read

Trust 21 - EmergingVerification L1Unclaimed author
29works
0followers
20topics
4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Identity and collaboration

How to connect with this researcher

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Log in to claim

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Research graph

See the researcher in context

Open full explorer

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

29 published item(s)

preprint2026arXiv

A Classification of Fractal Squares

Let $λ_K:\bbR^2\rightarrow\{0,1,\ldots\}\cup\{\infty\}$ be the lambda function of a planar comapctum $K$, as defined in MR4488162. It is known that a planar continuum is locally connected if and only if its lambda function vanishes everywhere, or equivalently, $λ_K(K)=\{0\}$. In this article we show that every fractal square $K$ satisfies $λ_K(K)\subset\{0,1\}$ and find criterions to classify when $λ_K(K)$ equals $\{0\}$, $\{1\}$ or $\{0,1\}$. Here for any integer $N\ge2$ and any set $\Dc=\left\{(i,j): 0\le i,j\le N-1\right\}$ with cardinality $\ge2$, if we set $K^{(0)}=[0,1]^2$ and $\displaystyle K^{(n)}=\left\{\frac{x+d}{N}: x\in K^{(n-1)}, d\in\Dc\right\}(n\ge1)$ then $K=\bigcap_nK^{(n)}$ is called a fractal square.

0 citations0 reviewsNo code linkedOpen access
preprint2026arXiv

Branch, or Layer? Zeroth-Order Optimization for Continual Learning of Vision-Language Models

Vision-Language Continual Learning (VLCL) has attracted significant research attention for its robust capabilities, and the adoption of Parameter-Efficient Fine-Tuning (PEFT) strategies is enabling these models to achieve competitive performance with substantially reduced resource consumption. However, dominated First-Order (FO) optimization is prone to trap models in suboptimal local minima, especially in limited exploration subspace within PEFT. To overcome this challenge, this paper pioneers a systematic exploration of adopting Zeroth-Order (ZO) optimization for PEFT-based VLCL. We first identify the incompatibility of naive full-ZO adoption in VLCL due to optimization process instability. We then investigate the application of ZO optimization from a modality branch-wise to a fine-grained layer-wise across various training units to identify an optimal strategy. Besides, a key theoretical insight reveals that vision modality exhibit higher variance than language counterparts in VLCL during the ZO optimization process, and we propose a modality-aware ZO strategy, which adopts gradient sign normalization in ZO and constrains vision modality perturbation to further improve performance. Benefiting from the adoption of ZO optimization, PEFT-based VLCL fulfills better ability to escape local minima during the optimization process, extensive experiments on four benchmarks demonstrate that our method achieves state-of-the-art results.

0 citations0 reviewsNo code linkedOpen access
preprint2026arXiv

Invisible Walls: Privacy-Preserving ISAC Empowered by Reconfigurable Intelligent Surfaces

The environmental and target-related information inherently carried in wireless signals, such as channel state information (CSI), has brought increasing attention to integrated sensing and communication (ISAC). However, it also raises pressing concerns about privacy leakage through eavesdropping. While existing efforts have attempted to mitigate this issue, they either fail to account for the needs of legitimate communication and sensing users or rely on hardware with high complexity and cost. To overcome these limitations, we propose PrivISAC, a plug-and-play, low-cost solution that leverages RIS to protect user privacy while preserving ISAC performance. At the core of PrivISAC is a novel strategy in which each RIS row is assigned two distinct beamforming vectors, from which we deliberately construct a limited set of RIS configurations. During operation, exactly one configuration is randomly activated at each time slot to introduce additional perturbations, effectively masking sensitive sensing information from unauthorized eavesdroppers. To jointly ensure privacy protection and communication performance, we design the two vectors such that their responses remain nearly identical in the communication direction, thereby preserving stable, high-throughput transmission, while exhibiting pronounced differences in the sensing direction, which introduces sufficient perturbations to thwart eavesdroppers. Additionally, to enable legitimate sensing under such randomized configurations, we introduce a time-domain masking and demasking method that allows the authorized receiver to associate each CSI sample with its underlying configuration and eliminate configuration-induced discrepancies, thereby recovering valid CSI. We implement PrivISAC on commodity wireless devices and experiment results show that PrivISAC provides strong privacy protection while preserving high-quality legitimate ISAC.

0 citations0 reviewsNo code linkedOpen access
preprint2026arXiv

SmoothCLAP: Soft-Target Enhanced Contrastive Language\--Audio Pretraining for Affective Computing

The ambiguity of human emotions poses several challenges for machine learning models, as they often overlap and lack clear delineating boundaries. Contrastive language-audio pretraining (CLAP) has emerged as a key technique for generalisable emotion recognition. However, as conventional CLAP enforces a strict one-to-one alignment between paired audio-text samples, it overlooks intra-modal similarity and treats all non-matching pairs as equally negative. This conflicts with the fuzzy boundaries between different emotions. To address this limitation, we propose SmoothCLAP, which introduces softened targets derived from intra-modal similarity and paralinguistic features. By combining these softened targets with conventional contrastive supervision, SmoothCLAP learns embeddings that respect graded emotional relationships, while retaining the same inference pipeline as CLAP. Experiments on eight affective computing tasks across English and German demonstrate that SmoothCLAP is consistently achieving superior performance. Our results highlight that leveraging soft supervision is a promising strategy for building emotion-aware audio-text models.

0 citations0 reviewsNo code linkedOpen access
preprint2026arXiv

TrojanPraise: Jailbreak LLMs via Benign Fine-Tuning

The demand of customized large language models (LLMs) has led to commercial LLMs offering black-box fine-tuning APIs, yet this convenience introduces a critical security loophole: attackers could jailbreak the LLMs by fine-tuning them with malicious data. Though this security issue has recently been exposed, the feasibility of such attacks is questionable as malicious training dataset is believed to be detectable by moderation models such as Llama-Guard-3. In this paper, we propose TrojanPraise, a novel finetuning-based attack exploiting benign and thus filter-approved data. Basically, TrojanPraise fine-tunes the model to associate a crafted word (e.g., "bruaf") with harmless connotations, then uses this word to praise harmful concepts, subtly shifting the LLM from refusal to compliance. To explain the attack, we decouple the LLM's internal representation of a query into two dimensions of knowledge and attitude. We demonstrate that successful jailbreak requires shifting the attitude while avoiding knowledge shift, a distortion in the model's understanding of the concept. To validate this attack, we conduct experiments on five opensource LLMs and two commercial LLMs under strict black-box settings. Results show that TrojanPraise achieves a maximum attack success rate of 95.88% while evading moderation.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Adapt to Adaptation: Learning Personalization for Cross-Silo Federated Learning

Conventional federated learning (FL) trains one global model for a federation of clients with decentralized data, reducing the privacy risk of centralized training. However, the distribution shift across non-IID datasets, often poses a challenge to this one-model-fits-all solution. Personalized FL aims to mitigate this issue systematically. In this work, we propose APPLE, a personalized cross-silo FL framework that adaptively learns how much each client can benefit from other clients' models. We also introduce a method to flexibly control the focus of training APPLE between global and local objectives. We empirically evaluate our method's convergence and generalization behaviors, and perform extensive experiments on two benchmark datasets and two medical imaging datasets under two non-IID settings. The results show that the proposed personalized FL framework, APPLE, achieves state-of-the-art performance compared to several other personalized FL approaches in the literature. The code is publicly available at https://github.com/ljaiverson/pFL-APPLE.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Connectedness and local cut points of generalized Sierpinski carpets

We investigate a homeomorphism problem on a class of self-similar sets called generalized Sierpinski carpets (or shortly GSCs). It follows from two well-known results by Hata and Whyburn that a connected GSC is homeomorphic to the standard Sierpinski carpet if and only if it has no local cut points. On the one hand, we show that to determine whether a given GSC is connected, it suffices to iterate the initial pattern twice. On the other hand, we obtain two criteria: (1) for a connected GSC to have cut points, (2) for a connected GSC with no cut points to have local cut points. With these two criteria, we characterize all GSCs that are homeomorphic to the standard Sierpinski carpet. Our results on cut points and local cut points hold for Baranski carpets, too. Moreover, we extend the connectedness result to Baranski sponges. Thus, we also characterize when a Baranski carpet is homeomorphic to the standard GSC.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Drawing Inductor Layout with a Reinforcement Learning Agent: Method and Application for VCO Inductors

Design of Voltage-Controlled Oscillator (VCO) inductors is a laborious and time-consuming task that is conventionally done manually by human experts. In this paper, we propose a framework for automating the design of VCO inductors, using Reinforcement Learning (RL). We formulate the problem as a sequential procedure, where wire segments are drawn one after another, until a complete inductor is created. We then employ an RL agent to learn to draw inductors that meet certain target specifications. In light of the need to tweak the target specifications throughout the circuit design cycle, we also develop a variant in which the agent can learn to quickly adapt to draw new inductors for moderately different target specifications. Our empirical results show that the proposed framework is successful at automatically generating VCO inductors that meet or exceed the target specification.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

How is Vaping Framed on Online Knowledge Dissemination Platforms?

We analyze 1,888 articles and 1,119,453 vaping posts to study how vaping is framed across multiple knowledge dissemination platforms (Wikipedia, Quora, Medium, Reddit, Stack Exchange, wikiHow). We use various NLP techniques to understand these differences. For example, n-grams, emotion recognition, and question answering results indicate that Medium, Quora, and Stack Exchange are appropriate venues for those looking to transition from smoking to vaping. Other platforms (Reddit, wikiHow) are more for vaping hobbyists and may not sufficiently dissuade youth vaping. Conversely, Wikipedia may exaggerate vaping harms, dissuading smokers from transitioning. A strength of our work is how the different techniques we have applied validate each other. Based on our results, we provide several recommendations. Stakeholders may utilize our findings to design informational tools to reinforce or mitigate vaping (mis)perceptions online.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

LatentFormer: Multi-Agent Transformer-Based Interaction Modeling and Trajectory Prediction

Multi-agent trajectory prediction is a fundamental problem in autonomous driving. The key challenges in prediction are accurately anticipating the behavior of surrounding agents and understanding the scene context. To address these problems, we propose LatentFormer, a transformer-based model for predicting future vehicle trajectories. The proposed method leverages a novel technique for modeling interactions among dynamic objects in the scene. Contrary to many existing approaches which model cross-agent interactions during the observation time, our method additionally exploits the future states of the agents. This is accomplished using a hierarchical attention mechanism where the evolving states of the agents autoregressively control the contributions of past trajectories and scene encodings in the final prediction. Furthermore, we propose a multi-resolution map encoding scheme that relies on a vision transformer module to effectively capture both local and global scene context to guide the generation of more admissible future trajectories. We evaluate the proposed method on the nuScenes benchmark dataset and show that our approach achieves state-of-the-art performance and improves upon trajectory metrics by up to 40%. We further investigate the contributions of various components of the proposed technique via extensive ablation studies.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Low Gain Avalanche Detectors with Good Time Resolution Developed by IHEP and IME for ATLAS HGTD project

This paper shows the simulation and test results of 50um thick Low Gain Avalanche Detectors (LGAD) sensors designed by the Institute of High Energy Physics (IHEP) and fabricated by the Institute of Microelectronics of the Chinese Academy of Sciences (IME). Three wafers have been produced with four different gain layer implant doses each. Different production processes, including variation in the n++ layer implant energy and carbon co-implantation were used. Test results show that the IHEP-IME sensors with the higher dose of gain layer have lower breakdown voltages and higher gain layer voltages from capacitance-voltage properties, which are consistent with the TCAD simulation. Beta test results show that the time resolution of IHEP-IME sensors is better than 35ps when operated at high voltage and the collected charges of IHEP-IME sensors are larger than 15fC before irradiation, which fulfill the required specifications of sensors before irradiations for the ATLAS HGTD project.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Offline Learning of Counterfactual Predictions for Real-World Robotic Reinforcement Learning

We consider real-world reinforcement learning (RL) of robotic manipulation tasks that involve both visuomotor skills and contact-rich skills. We aim to train a policy that maps multimodal sensory observations (vision and force) to a manipulator's joint velocities under practical considerations. We propose to use offline samples to learn a set of general value functions (GVFs) that make counterfactual predictions from the visual inputs. We show that combining the offline learned counterfactual predictions with force feedbacks in online policy learning allows efficient reinforcement learning given only a terminal (success/failure) reward. We argue that the learned counterfactual predictions form a compact and informative representation that enables sample efficiency and provides auxiliary reward signals that guide online explorations towards contact-rich states. Various experiments in simulation and real-world settings were performed for evaluation. Recordings of the real-world robot training can be found via https://sites.google.com/view/realrl.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Probing the Effect of Selection Bias on Generalization: A Thought Experiment

Learned systems in the domain of visual recognition and cognition impress in part because even though they are trained with datasets many orders of magnitude smaller than the full population of possible images, they exhibit sufficient generalization to be applicable to new and previously unseen data. Since training data sets typically represent small sampling of a domain, the possibility of bias in their composition is very real. But what are the limits of generalization given such bias, and up to what point might it be sufficient for a real problem task? Although many have examined issues regarding generalization, this question may require examining the data itself. Here, we focus on the characteristics of the training data that may play a role. Other disciplines have grappled with these problems, most interestingly epidemiology, where experimental bias is a critical concern. The range and nature of data biases seen clinically are really quite relatable to learned vision systems. One obvious way to deal with bias is to ensure a large enough training set, but this might be infeasible for many domains. Another approach might be to perform a statistical analysis of the actual training set, to determine if all aspects of the domain are fairly captured. This too is difficult, in part because the full set of variables might not be known, or perhaps not even knowable. Here, we try a different approach in the tradition of the Thought Experiment, whose most famous instance may be Schrödinger's Cat. There are many types of bias as will be seen, but we focus only on one, selection bias. The point of the thought experiment is not to demonstrate problems with all learned systems. Rather, this might be a simple theoretical tool to probe into bias during data collection to highlight deficiencies that might then deserve extra attention either in data collection or system development.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Understanding and Mitigating the Limitations of Prioritized Experience Replay

Prioritized Experience Replay (ER) has been empirically shown to improve sample efficiency across many domains and attracted great attention; however, there is little theoretical understanding of why such prioritized sampling helps and its limitations. In this work, we take a deep look at the prioritized ER. In a supervised learning setting, we show the equivalence between the error-based prioritized sampling method for mean squared error and uniform sampling for cubic power loss. We then provide theoretical insight into why it improves convergence rate upon uniform sampling during early learning. Based on the insight, we further point out two limitations of the prioritized ER method: 1) outdated priorities and 2) insufficient coverage of the sample space. To mitigate the limitations, we propose our model-based stochastic gradient Langevin dynamics sampling method. We show that our method does provide states distributed close to an ideal prioritized sampling distribution estimated by the brute-force method, which does not suffer from the two limitations. We conduct experiments on both discrete and continuous control problems to show our approach's efficacy and examine the practical implication of our method in an autonomous driving application.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

What makes useful auxiliary tasks in reinforcement learning: investigating the effect of the target policy

Auxiliary tasks have been argued to be useful for representation learning in reinforcement learning. Although many auxiliary tasks have been empirically shown to be effective for accelerating learning on the main task, it is not yet clear what makes useful auxiliary tasks. Some of the most promising results are on the pixel control, reward prediction, and the next state prediction auxiliary tasks; however, the empirical results are mixed, showing substantial improvements in some cases and marginal improvements in others. Careful investigations of how auxiliary tasks help the learning of the main task is necessary. In this paper, we take a step studying the effect of the target policies on the usefulness of the auxiliary tasks formulated as general value functions. General value functions consist of three core elements: 1) policy 2) cumulant 3) continuation function. Our focus on the role of the target policy of the auxiliary tasks is motivated by the fact that the target policy determines the behavior about which the agent wants to make a prediction and the state-action distribution that the agent is trained on, which further affects the main task learning. Our study provides insights about questions such as: Does a greedy policy result in bigger improvement gains compared to other policies? Is it best to set the auxiliary task policy to be the same as the main task policy? Does the choice of the target policy have a substantial effect on the achieved performance gain or simple strategies for setting the policy, such as using a uniformly random policy, work as well? Our empirical results suggest that: 1) Auxiliary tasks with the greedy policy tend to be useful. 2) Most policies, including a uniformly random policy, tend to improve over the baseline. 3) Surprisingly, the main task policy tends to be less useful compared to other policies.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

CoachNet: An Adversarial Sampling Approach for Reinforcement Learning

Despite the recent successes of reinforcement learning in games and robotics, it is yet to become broadly practical. Sample efficiency and unreliable performance in rare but challenging scenarios are two of the major obstacles. Drawing inspiration from the effectiveness of deliberate practice for achieving expert-level human performance, we propose a new adversarial sampling approach guided by a failure predictor named "CoachNet". CoachNet is trained online along with the agent to predict the probability of failure. This probability is then used in a stochastic sampling process to guide the agent to more challenging episodes. This way, instead of wasting time on scenarios that the agent has already mastered, training is focused on the agent's "weak spots". We present the design of CoachNet, explain its underlying principles, and empirically demonstrate its effectiveness in improving sample efficiency and test-time robustness in common continuous control tasks.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems

Multiagent reinforcement learning (MARL) has achieved a remarkable amount of success in solving various types of video games. A cornerstone of this success is the auto-curriculum framework, which shapes the learning process by continually creating new challenging tasks for agents to adapt to, thereby facilitating the acquisition of new skills. In order to extend MARL methods to real-world domains outside of video games, we envision in this blue sky paper that maintaining a diversity-aware auto-curriculum is critical for successful MARL applications. Specifically, we argue that \emph{behavioural diversity} is a pivotal, yet under-explored, component for real-world multiagent learning systems, and that significant work remains in understanding how to design a diversity-aware auto-curriculum. We list four open challenges for auto-curriculum techniques, which we believe deserve more attention from this community. Towards validating our vision, we recommend modelling realistic interactive behaviours in autonomous driving as an important test bed, and recommend the SMARTS/ULTRA benchmark.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Open-set Intersection Intention Prediction for Autonomous Driving

Intention prediction is a crucial task for Autonomous Driving (AD). Due to the variety of size and layout of intersections, it is challenging to predict intention of human driver at different intersections, especially unseen and irregular intersections. In this paper, we formulate the prediction of intention at intersections as an open-set prediction problem that requires context specific matching of the target vehicle state and the diverse intersection configurations that are in principle unbounded. We capture map-centric features that correspond to intersection structures under a spatial-temporal graph representation, and use two MAAMs (mutually auxiliary attention module) that cover respectively lane-level and exitlevel intentions to predict a target that best matches intersection elements in map-centric feature space. Under our model, attention scores estimate the probability distribution of the openset intentions that are contextually defined by the structure of the current intersection. The proposed model is trained and evaluated on simulated dataset. Furthermore, the model, trained on simulated dataset and without any fine tuning, is directly validated on in-house real-world dataset collected at 98 realworld intersections and exhibits satisfactory performance,demonstrating the practical viability of our approach.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Review on Ranking and Selection: A New Perspective

In this paper, we briefly review the development of ranking-and-selection (R&S) in the past 70 years, especially the theoretical achievements and practical applications in the last 20 years. Different from the frequentist and Bayesian classifications adopted by Kim and Nelson (2006b) and Chick (2006) in their review articles, we categorize existing R&S procedures into fixed-precision and fixed-budget procedures, as in Hunter and Nelson (2017). We show that these two categories of procedures essentially differ in the underlying methodological formulations, i.e., they are built on hypothesis testing and dynamic-programming, respectively. In light of this variation, we review in detail some well-known procedures in the literature and show how they fit into these two formulations. In addition, we discuss the use of R&S procedures in solving various practical problems and propose what we think are the important research questions in the field.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Adaptive Hierarchical Down-Sampling for Point Cloud Classification

While several convolution-like operators have recently been proposed for extracting features out of point clouds, down-sampling an unordered point cloud in a deep neural network has not been rigorously studied. Existing methods down-sample the points regardless of their importance for the output. As a result, some important points in the point cloud may be removed, while less valuable points may be passed to the next layers. In contrast, adaptive down-sampling methods sample the points by taking into account the importance of each point, which varies based on the application, task and training data. In this paper, we propose a permutation-invariant learning-based adaptive down-sampling layer, called Critical Points Layer (CPL), which reduces the number of points in an unordered point cloud while retaining the important points. Unlike most graph-based point cloud down-sampling methods that use $k$-NN search algorithm to find the neighbouring points, CPL is a global down-sampling method, rendering it computationally very efficient. The proposed layer can be used along with any graph-based point cloud convolution layer to form a convolutional neural network, dubbed CP-Net in this paper. We introduce a CP-Net for $3$D object classification that achieves the best accuracy for the ModelNet$40$ dataset among point cloud-based methods, which validates the effectiveness of the CPL.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Enhancing Intrinsic Adversarial Robustness via Feature Pyramid Decoder

Whereas adversarial training is employed as the main defence strategy against specific adversarial samples, it has limited generalization capability and incurs excessive time complexity. In this paper, we propose an attack-agnostic defence framework to enhance the intrinsic robustness of neural networks, without jeopardizing the ability of generalizing clean samples. Our Feature Pyramid Decoder (FPD) framework applies to all block-based convolutional neural networks (CNNs). It implants denoising and image restoration modules into a targeted CNN, and it also constraints the Lipschitz constant of the classification layer. Moreover, we propose a two-phase strategy to train the FPD-enhanced CNN, utilizing $ε$-neighbourhood noisy images with multi-task and self-supervised learning. Evaluated against a variety of white-box and black-box attacks, we demonstrate that FPD-enhanced CNNs gain sufficient robustness against general adversarial samples on MNIST, SVHN and CALTECH. In addition, if we further conduct adversarial training, the FPD-enhanced CNNs perform better than their non-enhanced versions.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

LISPR: An Options Framework for Policy Reuse with Reinforcement Learning

We propose a framework for transferring any existing policy from a potentially unknown source MDP to a target MDP. This framework (1) enables reuse in the target domain of any form of source policy, including classical controllers, heuristic policies, or deep neural network-based policies, (2) attains optimality under suitable theoretical conditions, and (3) guarantees improvement over the source policy in the target MDP. These are achieved by packaging the source policy as a black-box option in the target MDP and providing a theoretically grounded way to learn the option's initiation set through general value functions. Our approach facilitates the learning of new policies by (1) maximizing the target MDP reward with the help of the black-box option, and (2) returning the agent to states in the learned initiation set of the black-box option where it is already optimal. We show that these two variants are equivalent in performance under some conditions. Through a series of experiments in simulated environments, we demonstrate that our framework performs excellently in sparse reward problems given (sub-)optimal source policies and improves upon prior art in transfer methods such as continual learning and progressive networks, which lack our framework's desirable theoretical properties.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Neighborhood Cognition Consistent Multi-Agent Reinforcement Learning

Social psychology and real experiences show that cognitive consistency plays an important role to keep human society in order: if people have a more consistent cognition about their environments, they are more likely to achieve better cooperation. Meanwhile, only cognitive consistency within a neighborhood matters because humans only interact directly with their neighbors. Inspired by these observations, we take the first step to introduce \emph{neighborhood cognitive consistency} (NCC) into multi-agent reinforcement learning (MARL). Our NCC design is quite general and can be easily combined with existing MARL methods. As examples, we propose neighborhood cognition consistent deep Q-learning and Actor-Critic to facilitate large-scale multi-agent cooperations. Extensive experiments on several challenging tasks (i.e., packet routing, wifi configuration, and Google football player control) justify the superior performance of our methods compared with state-of-the-art MARL approaches.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

NMR and NQR studies on transition-metal arsenide superconductors LaRu2As2, KCa2Fe4As4F2, and A2Cr3As3

We report 75As-nuclear magnetic resonance (NMR) and nuclear quadrupole resonance (NQR) measurements on transition-metal arsenides LaRu2As2, KCa2Fe4As4F2, and A2Cr3As3. In the superconducting state of LaRu2As2, a Hebel- Slichter coherence peak is found in the temperature dependence of the spin-lattice relaxation rate-1/T1 just below Tc, which indicates that LaRu2As2 is a full-gap superperconducor. For KCa2Fe4As4F2, antiferromagnetic spin fluctuations are observed in the normal state. We further find that the anisotropy rate RAF = Tc1/Tab1 is small and temperature independent, implying that the low energy spin fluctuations are isotropic in spin space. Our results indicate that KCa2Fe4As4F2 is a moderately overdoped iron-arsenide high-temperature superconductor with a stoichiometric composition. For A2Cr3As3, we calculate the electric field gradient by first-principle method and assign the 75As-NQR peaks with two crystallographically different As sites, paving the way for further NMR investigation.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Optimizing orbits for TianQin

TianQin is a geocentric space-based gravitational-wave observatory mission consisting of three drag-free controlled satellites in an equilateral triangle with an orbital radius of $ 10^{5}$ km. The constellation faces the white-dwarf binary RX J0806.3+1527 located slightly below the ecliptic plane, and is subject to gravitational perturbations that can distort the formation. In this study, we present combined methods to optimize the TianQin orbits so that a set of 5-year stability requirements can be met. Moreover, we discuss slow long-term drift of the detector pointing due to orbital precession, and put forward stable orbits with six other pointings along the lunar orbital plane. Some implications of the findings are pointed out.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

The First Round Result from the TianQin-1 Satellite

The TianQin-1 satellite (TQ-1), which is the first technology demonstration satellite for the TianQin project, was launched on 20 December 2019. The first round of experiment had been carried out from 21 December 2019 until 1 April 2020. The residual acceleration of the satellite is found to be about $1\times10^{-10}~{\rm m}/{\rm s}^{2}/{\rm Hz}^{1/2}$ at $0.1~{\rm Hz}\,$ and about $5\times10^{-11}~{\rm m}/{\rm s}^{2}/{\rm Hz}^{1/2}$ at $0.05~{\rm Hz}\,$, measured by an inertial sensor with a sensitivity of $5\times10^{-12}~{\rm m}/{\rm s}^{2}/{\rm Hz}^{1/2}$ at $0.1~{\rm Hz}\,$. The micro-Newton thrusters has demonstrated a thrust resolution of $0.1~μ{\rm N}$ and a thrust noise of $0.3~μ{\rm N}/{\rm Hz}^{1/2}$ at $0.1~{\rm Hz}$. The residual noise of the satellite with drag-free control is $3\times10^{-9}~{\rm m}/{\rm s}^{2}/{\rm Hz}^{1/2}$ at $0.1~{\rm Hz}\,$. The noise level of the optical readout system is about $30~{\rm pm}/{\rm Hz}^{1/2}$ at $0.1~{\rm Hz}\,$. The temperature stability at temperature monitoring position is controlled to be about $\pm3~{\rm mK}$ per orbit, and the mismatch between the center-of-mass of the satellite and that of the test mass is measured with a precision of better than $0.1~{\rm mm}$.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

The TianQin project: current progress on science and technology

TianQin is a planned space-based gravitational wave (GW) observatory consisting of three earth orbiting satellites with an orbital radius of about $10^5~{\rm km}$. The satellites will form a equilateral triangle constellation the plane of which is nearly perpendicular to the ecliptic plane. TianQin aims to detect GWs between $10^{-4}~{\rm Hz}$ and $1~{\rm Hz}$ that can be generated by a wide variety of important astrophysical and cosmological sources, including the inspiral of Galactic ultra-compact binaries, the inspiral of stellar-mass black hole binaries, extreme mass ratio inspirals, the merger of massive black hole binaries, and possibly the energetic processes in the very early universe or exotic sources such as cosmic strings. In order to start science operations around 2035, a roadmap called the 0123 plan is being used to bring the key technologies of TianQin to maturity, supported by the construction of a series of research facilities on the ground. Two major projects of the 0123 plan are being carried out. In this process, the team has created a new generation $17~{\rm cm}$ single-body hollow corner-cube retro-reflector which has been launched with the QueQiao satellite on 21 May 2018; a new laser ranging station equipped with a $1.2~{\rm m}$ telescope has been constructed and the station has successfully ranged to all the five retro-reflectors on the Moon; and the TianQin-1 experimental satellite has been launched on 20 December 2019 and the first round result shows that the satellite has exceeded all of its mission requirements.

0 citations0 reviewsNo code linkedOpen access
preprint2019arXiv

ZAIGA: Zhaoshan Long-baseline Atom Interferometer Gravitation Antenna

The Zhaoshan long-baseline Atom Interferometer Gravitation Antenna (ZAIGA) is a new type of underground laser-linked interferometer facility, and is currently under construction. It is in the 200-meter-on-average underground of a mountain named Zhaoshan which is about 80 km southeast to Wuhan. ZAIGA will be equipped with long-baseline atom interferometers, high-precision atom clocks, and large-scale gyros. ZAIGA facility will take an equilateral triangle configuration with two 1-km-apart atom interferometers in each arm, a 300-meter vertical tunnel with atom fountain and atom clocks mounted, and a tracking-and-ranging 1-km-arm-length prototype with lattice optical clocks linked by locked lasers. The ZAIGA facility will be used for experimental research on gravitation and related problems including gravitational wave detection, high-precision test of the equivalence principle of micro-particles, clock based gravitational red-shift measurement, rotation measurement and gravito-magnetic effect.

0 citations0 reviewsNo code linkedOpen access
preprint2018arXiv

A unified theory of grain growth in polycrystalline materials

Grain growth is a ubiquitous and fundamental phenomenon observed in the cellular structures with the grain assembly separated by a network of grain boundaries, including metals and ceramics. However, the underlying mechanism of grain growth has remained ambiguous for more than 60 years. The models for grain growth, based on the classically linear relationship between the grain boundary migration and capillary driving force, generally predict normal grain growth. Quantitative model for abnormal grain growth is lacking despite decades of efforts. Here, we present a unified model to reveal quantitatively how grain growth evolves, which predicts the normal, abnormal and stagnant behaviors of grain growth in polycrystalline materials. Our model indicates that the relationship between grain boundary migration and capillary driving force is generally nonlinear, but will switch to be the classically linear relationship in a specific case. Furthermore, the grain growth experiments observed in polycrystalline SrTiO3 demonstrates the validity of the unified model. Our study provides a unified, quantitative model to understand and predict grain growth in polycrystalline materials, and thus offers helpful guides for the microstructural design to optimize the properties of polycrystalline materials.

0 citations0 reviewsNo code linkedOpen access