Trust snapshot

Quick read

Trust 21 - EmergingVerification L1Unclaimed author
123works
0followers
48topics
4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Identity and collaboration

How to connect with this researcher

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Log in to claim

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Research graph

See the researcher in context

Open full explorer

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

123 published item(s)

preprint2026arXiv

Hypergraph and Latent ODE Learning for Multimodal Root Cause Localization in Microservices

Root cause localization in cloud native microservice systems requires modeling complex service dependencies, irregular temporal dynamics, and heterogeneous observability data. We present HyperODE RCA, a unified framework that combines hypergraph attention learning, latent ordinary differential equations, and multimodal cross attention fusion for fine grained root cause analysis. The method learns higher order service interactions through differentiable hyperedge construction, captures continuous anomaly evolution from irregular observations with an ODE RNN encoder, and adaptively fuses logs, traces, metrics, entities, and events using context aware modality routing. We further improve robustness with a variational information bottleneck, temporal causal regularization, and invariant risk constraints. Experiments on the Tianchi AIOps benchmark show clear gains over strong baselines in ranking and classification performance, while preserving interpretability through learned hypergraph attention.

preprint2024arXiv

An Inexact Preconditioned Zeroth-order Proximal Method for Composite Optimization

In this paper, we consider the composite optimization problem, where the objective function integrates a continuously differentiable loss function with a nonsmooth regularization term. Moreover, only the function values for the differentiable part of the objective function are available. To efficiently solve this composite optimization problem, we propose a preconditioned zeroth-order proximal gradient method in which the gradients and preconditioners are estimated by finite-difference schemes based on the function values at the same trial points. We establish the global convergence and worst-case complexity for our proposed method. Numerical experiments exhibit the superiority of our developed method.

preprint2024arXiv

DeepSeek LLM: Scaling Open-Source Language Models with Longtermism

The rapid development of open-source large language models (LLMs) has been truly remarkable. However, the scaling law described in previous literature presents varying conclusions, which casts a dark cloud over scaling LLMs. We delve into the study of scaling laws and present our distinctive findings that facilitate scaling of large scale models in two commonly used open-source configurations, 7B and 67B. Guided by the scaling laws, we introduce DeepSeek LLM, a project dedicated to advancing open-source language models with a long-term perspective. To support the pre-training phase, we have developed a dataset that currently consists of 2 trillion tokens and is continuously expanding. We further conduct supervised fine-tuning (SFT) and Direct Preference Optimization (DPO) on DeepSeek LLM Base models, resulting in the creation of DeepSeek Chat models. Our evaluation results demonstrate that DeepSeek LLM 67B surpasses LLaMA-2 70B on various benchmarks, particularly in the domains of code, mathematics, and reasoning. Furthermore, open-ended evaluations reveal that DeepSeek LLM 67B Chat exhibits superior performance compared to GPT-3.5.

preprint2024arXiv

Enhancing RBF-FD Efficiency for Highly Non-Uniform Node Distributions via Adaptivity

Radial basis function generated finite-difference (RBF-FD) methods have recently gained popularity due to their flexibility with irregular node distributions. However, the convergence theories in the literature, when applied to nonuniform node distributions, require shrinking fill distance and do not take advantage of areas with high data density. Non-adaptive approach using same stencil size and degree of appended polynomial will have higher local accuracy at high density region, but has no effect on the overall order of convergence and could be a waste of computational power. This work proposes an adaptive RBF-FD method that utilizes the local data density to achieve a desirable order accuracy. By performing polynomial refinement and using adaptive stencil size based on data density, the adaptive RBF-FD method yields differentiation matrices with higher sparsity while achieving the same user-specified convergence order for nonuniform point distributions. This allows the method to better leverage regions with higher node density, maintaining both accuracy and efficiency compared to standard non-adaptive RBF-FD methods.

preprint2024arXiv

Local well-posedness and global stability of one-dimensional shallow water equations with surface tension and constant contact angle

We consider the one-dimensional shallow water problem with capillary surfaces and moving contact {lines}. An energy-based model is derived from the two-dimensional water wave equations, where we explicitly discuss the case of a stationary force balance at a moving contact line and highlight necessary changes to consider dynamic contact angles. The moving contact line becomes our free boundary at the level of shallow water equations, and the depth of the shallow water degenerates near the free boundary, which causes singularities for the derivatives and degeneracy for the viscosity. This is similar to the physical vacuum of compressible flows in the literature. The equilibrium, the global stability of the equilibrium, and the local well-posedness theory are established in this paper.

preprint2023arXiv

A Constraint Dissolving Approach for Nonsmooth Optimization over the Stiefel Manifold

This paper focus on the minimization of a possibly nonsmooth objective function over the Stiefel manifold. The existing approaches either lack efficiency or can only tackle prox-friendly objective functions. We propose a constraint dissolving function named NCDF and show that it has the same first-order stationary points and local minimizers as the original problem in a neighborhood of the Stiefel manifold. Furthermore, we show that the Clarke subdifferential of NCDF is easy to achieve from the Clarke subdifferential of the objective function. Therefore, various existing approaches for unconstrained nonsmooth optimization can be directly applied to nonsmooth optimization problems over the Stiefel manifold. We propose a framework for developing subgradient-based methods and establish their convergence properties based on prior works. Furthermore, based on our proposed framework, we can develop efficient approaches for optimization over the Stiefel manifold. Preliminary numerical experiments further highlight that the proposed constraint dissolving approach yields efficient and direct implementations of various unconstrained approaches to nonsmooth optimization problems over the Stiefel manifold.

preprint2023arXiv

An Overview of Human Activity Recognition Using Wearable Sensors: Healthcare and Artificial Intelligence

With the rapid development of the internet of things (IoT) and artificial intelligence (AI) technologies, human activity recognition (HAR) has been applied in a variety of domains such as security and surveillance, human-robot interaction, and entertainment. Even though a number of surveys and review papers have been published, there is a lack of HAR overview papers focusing on healthcare applications that use wearable sensors. Therefore, we fill in the gap by presenting this overview paper. In particular, we present our projects to illustrate the system design of HAR applications for healthcare. Our projects include early mobility identification of human activities for intensive care unit (ICU) patients and gait analysis of Duchenne muscular dystrophy (DMD) patients. We cover essential components of designing HAR systems including sensor factors (e.g., type, number, and placement location), AI model selection (e.g., classical machine learning models versus deep learning models), and feature engineering. In addition, we highlight the challenges of such healthcare-oriented HAR systems and propose several research opportunities for both the medical and the computer science community.

preprint2023arXiv

Chiral Topological superconductivity in the OAI/SC/FMI heterostructure avoiding the subband problem

Implementing topological superconductivity (TSC) and Majorana states (MSs) is one of the most significant and challenging tasks in both fundamental physics and topological quantum computations. In this work, taking the obstructed atomic insulator (OAI) Nb3Br8, s-wave superconductor (SC) NbSe2 and ferromagnetic insulator (FMI) as example, we propose a new setup to realize the 2D chiral TSC and MSs in the OAI/SC/FMI heterostructure, which could avoid the subband problem effectively and has the advantage of huge Rashba spin-orbit coupling. As a result, the TSC phase can be stabilized in a wide region of chemical potential and Zeeman field, and four distinct TSC phases with superconducting Chern number N= -1, -2, -3, 3 can be achieved. Moreover, a 2D BdG Hamiltonian based on the triangular lattice of obstructed Wannier charge centers, combined with the s-wave superconductivity paring and Zeeman field, is constructed to understand the whole topological phase diagram analytically. These results expand the application of OAIs and pave a new way to realize the TSC and MSs with unique advantages.

preprint2023arXiv

Dual-Stream Diffusion Net for Text-to-Video Generation

With the emerging diffusion models, recently, text-to-video generation has aroused increasing attention. But an important bottleneck therein is that generative videos often tend to carry some flickers and artifacts. In this work, we propose a dual-stream diffusion net (DSDN) to improve the consistency of content variations in generating videos. In particular, the designed two diffusion streams, video content and motion branches, could not only run separately in their private spaces for producing personalized video variations as well as content, but also be well-aligned between the content and motion domains through leveraging our designed cross-transformer interaction module, which would benefit the smoothness of generated videos. Besides, we also introduce motion decomposer and combiner to faciliate the operation on video motion. Qualitative and quantitative experiments demonstrate that our method could produce amazing continuous videos with fewer flickers.

preprint2023arXiv

On the boundary layer arising from fast internal waves dynamics

In this paper, we investigate the boundary layer arising from the fast internal waves in the Boussinesq equations with the Brunt-Vaisälä frequency of order $ \mathcal O(1/\varepsilon) $. For the homogeneous-in-height stratification, previous work by \emph{Desjardins, Lannes, Saut, 3(1):153--192, Water Waves, 2021} establishes uniform-in-$ε$ estimates locally in time, with additional constraints on the boundary data initially, which essentially restricts the dynamics in the spatially periodic domain. Removing such constraints, our goal is to investigate the general near-boundary behavior. We observe that the fast internal waves will give rise to large growth of the spatial derivatives in the normal direction of the solutions in the vicinity of the boundary. To capture this phenomenon, we introduce an inviscid boundary layer using a natural scaling. In addition, we investigate the well-posedness of such a boundary layer system in the space of analytic functions.

preprint2023arXiv

Smoothing Gradient Tracking for Decentralized Optimization over the Stiefel Manifold with Non-smooth Regularizers

Recently, decentralized optimization over the Stiefel manifold has attacked tremendous attentions due to its wide range of applications in various fields. Existing methods rely on the gradients to update variables, which are not applicable to the objective functions with non-smooth regularizers, such as sparse PCA. In this paper, to the best of our knowledge, we propose the first decentralized algorithm for non-smooth optimization over Stiefel manifolds. Our algorithm approximates the non-smooth part of objective function by its Moreau envelope, and then existing algorithms for smooth optimization can be deployed. We establish the convergence guarantee with the iteration complexity of $\mathcal{O} (ε^{-4})$. Numerical experiments conducted under the decentralized setting demonstrate the effectiveness and efficiency of our algorithm.

preprint2022arXiv

A Communication-Efficient and Privacy-Aware Distributed Algorithm for Sparse PCA

Sparse principal component analysis (PCA) improves interpretability of the classic PCA by introducing sparsity into the dimension-reduction process. Optimization models for sparse PCA, however, are generally non-convex, non-smooth and more difficult to solve, especially on large-scale datasets requiring distributed computation over a wide network. In this paper, we develop a distributed and centralized algorithm called DSSAL1 for sparse PCA that aims to achieve low communication overheads by adapting a newly proposed subspace-splitting strategy to accelerate convergence. Theoretically, convergence to stationary points is established for DSSAL1. Extensive numerical results show that DSSAL1 requires far fewer rounds of communication than state-of-the-art peer methods. In addition, we make the case that since messages exchanged in DSSAL1 are well-masked, the possibility of private-data leakage in DSSAL1 is much lower than in some other distributed algorithms.

preprint2022arXiv

A Convergence Analysis of Nesterov's Accelerated Gradient Method in Training Deep Linear Neural Networks

Momentum methods, including heavy-ball~(HB) and Nesterov's accelerated gradient~(NAG), are widely used in training neural networks for their fast convergence. However, there is a lack of theoretical guarantees for their convergence and acceleration since the optimization landscape of the neural network is non-convex. Nowadays, some works make progress towards understanding the convergence of momentum methods in an over-parameterized regime, where the number of the parameters exceeds that of the training instances. Nonetheless, current results mainly focus on the two-layer neural network, which are far from explaining the remarkable success of the momentum methods in training deep neural networks. Motivated by this, we investigate the convergence of NAG with constant learning rate and momentum parameter in training two architectures of deep linear networks: deep fully-connected linear neural networks and deep linear ResNets. Based on the over-parameterization regime, we first analyze the residual dynamics induced by the training trajectory of NAG for a deep fully-connected linear neural network under the random Gaussian initialization. Our results show that NAG can converge to the global minimum at a $(1 - \mathcal{O}(1/\sqrtκ))^t$ rate, where $t$ is the iteration number and $κ> 1$ is a constant depending on the condition number of the feature matrix. Compared to the $(1 - \mathcal{O}(1/κ))^t$ rate of GD, NAG achieves an acceleration over GD. To the best of our knowledge, this is the first theoretical guarantee for the convergence of NAG to the global minimum in training deep neural networks. Furthermore, we extend our analysis to deep linear ResNets and derive a similar convergence result.

preprint2022arXiv

A Variance-Reduced Stochastic Gradient Tracking Algorithm for Decentralized Optimization with Orthogonality Constraints

Decentralized optimization with orthogonality constraints is found widely in scientific computing and data science. Since the orthogonality constraints are nonconvex, it is quite challenging to design efficient algorithms. Existing approaches leverage the geometric tools from Riemannian optimization to solve this problem at the cost of high sample and communication complexities. To relieve this difficulty, based on two novel techniques that can waive the orthogonality constraints, we propose a variance-reduced stochastic gradient tracking (VRSGT) algorithm with the convergence rate of $O(1 / k)$ to a stationary point. To the best of our knowledge, VRSGT is the first algorithm for decentralized optimization with orthogonality constraints that reduces both sampling and communication complexities simultaneously. In the numerical experiments, VRSGT has a promising performance in a real-world autonomous driving application.

preprint2022arXiv

Admission Control for Double-ended Queues

We consider a controlled double-ended queue consisting of two classes of customers, labeled sellers and buyers. The sellers and buyers arrive in a trading market according to two independent renewal processes. Whenever there is a seller and buyer pair, they are matched and leave the system instantaneously. The matching follows first-come-first-match service discipline. Those customers who cannot be matched immediately need to wait in the designated queue, and they are assumed to be impatient with generally distributed patience times. The control problem is concerned with the trade-off between blocking and abandonment for each class and the interplay of statistical behaviors of the two classes, and its objective is to choose optimal queue-capacities (buffer lengths) for sellers and buyers to minimize an infinite horizon discounted linear cost functional which consists of holding costs and penalty costs for blocking and abandonment. When the arrival intensities of both customer classes tend to infinity in concert, we use a heavy traffic approximation to formulate an approximate diffusion control problem (DCP), and develop an optimal threshold policy for the DCP. Finally, we employ the DCP solution to establish an easy-to-implement asymptotically optimal threshold policy for the original queueing control problem.

preprint2022arXiv

An Efficient Subpopulation-based Membership Inference Attack

Membership inference attacks allow a malicious entity to predict whether a sample is used during training of a victim model or not. State-of-the-art membership inference attacks have shown to achieve good accuracy which poses a great privacy threat. However, majority of SOTA attacks require training dozens to hundreds of shadow models to accurately infer membership. This huge computation cost raises questions about practicality of these attacks on deep models. In this paper, we introduce a fundamentally different MI attack approach which obviates the need to train hundreds of shadow models. Simply put, we compare the victim model output on the target sample versus the samples from the same subpopulation (i.e., semantically similar samples), instead of comparing it with the output of hundreds of shadow models. The intuition is that the model response should not be significantly different between the target sample and its subpopulation if it was not a training sample. In cases where subpopulation samples are not available to the attacker, we show that training only a single generative model can fulfill the requirement. Hence, we achieve the state-of-the-art membership inference accuracy while significantly reducing the training computation cost.

preprint2022arXiv

An Inexact Augmented Lagrangian Algorithm for Training Leaky ReLU Neural Network with Group Sparsity

The leaky ReLU network with a group sparse regularization term has been widely used in the recent years. However, training such a network yields a nonsmooth nonconvex optimization problem and there exists a lack of approaches to compute a stationary point deterministically. In this paper, we first resolve the multi-layer composite term in the original optimization problem by introducing auxiliary variables and additional constraints. We show the new model has a nonempty and bounded solution set and its feasible set satisfies the Mangasarian-Fromovitz constraint qualification. Moreover, we show the relationship between the new model and the original problem. Remarkably, we propose an inexact augmented Lagrangian algorithm for solving the new model and show the convergence of the algorithm to a KKT point. Numerical experiments demonstrate that our algorithm is more efficient for training sparse leaky ReLU neural networks than some well-known algorithms.

preprint2022arXiv

ASER: Towards Large-scale Commonsense Knowledge Acquisition via Higher-order Selectional Preference over Eventualities

Commonsense knowledge acquisition and reasoning have long been a core artificial intelligence problem. However, in the past, there has been a lack of scalable methods to collect commonsense knowledge. In this paper, we propose to develop principles for collecting commonsense knowledge based on selectional preference. We generalize the definition of selectional preference from one-hop linguistic syntactic relations to higher-order relations over linguistic graphs. Unlike previous commonsense knowledge definition (e.g., ConceptNet), selectional preference (SP) knowledge only relies on statistical distribution over linguistic graphs, which can be efficiently and accurately acquired from the unlabeled corpus with modern tools. Following this principle, we develop a large-scale eventuality (a linguistic term covering activity, state, and event)-based knowledge graph ASER, where each eventuality is represented as a dependency graph, and the relation between them is a discourse relation defined in shallow discourse parsing. The higher-order selectional preference over collected linguistic graphs reflects various kinds of commonsense knowledge. Moreover, motivated by the observation that humans understand events by abstracting the observed events to a higher level and can thus transfer their knowledge to new events, we propose a conceptualization module to significantly boost the coverage of ASER. In total, ASER contains 648 million edges between 438 million eventualities. After conceptualization with Probase, a selectional preference based concept-instance relational knowledge base, our concept graph contains 15 million conceptualized eventualities and 224 million edges between them. Detailed analysis is provided to demonstrate its quality. All the collected data, APIs, and tools are available at https://github.com/HKUST-KnowComp/ASER.

preprint2022arXiv

Autonomous Smart Grid Fault Detection

Smart grid plays a crucial role for the smart society and the upcoming carbon neutral society. Achieving autonomous smart grid fault detection is critical for smart grid system state awareness, maintenance and operation. This paper focuses on fault monitoring in smart grid and discusses the inherent technical challenges and solutions. In particular, we first present the basic principles of smart grid fault detection. Then, we explain the new requirements for autonomous smart grid fault detection, the technical challenges and their possible solutions. A case study is introduced, as a preliminary study for autonomous smart grid fault detection. In addition, we highlight relevant directions for future research.

preprint2022arXiv

Boosting Graph Structure Learning with Dummy Nodes

With the development of graph kernels and graph representation learning, many superior methods have been proposed to handle scalability and oversmoothing issues on graph structure learning. However, most of those strategies are designed based on practical experience rather than theoretical analysis. In this paper, we use a particular dummy node connecting to all existing vertices without affecting original vertex and edge properties. We further prove that such the dummy node can help build an efficient monomorphic edge-to-vertex transform and an epimorphic inverse to recover the original graph back. It also indicates that adding dummy nodes can preserve local and global structures for better graph representation learning. We extend graph kernels and graph neural networks with dummy nodes and conduct experiments on graph classification and subgraph isomorphism matching tasks. Empirical results demonstrate that taking graphs with dummy nodes as input significantly boosts graph structure learning, and using their edge-to-vertex graphs can also achieve similar results. We also discuss the gain of expressive power from the dummy in neural networks.

preprint2022arXiv

Brain-inspired Graph Spiking Neural Networks for Commonsense Knowledge Representation and Reasoning

How neural networks in the human brain represent commonsense knowledge, and complete related reasoning tasks is an important research topic in neuroscience, cognitive science, psychology, and artificial intelligence. Although the traditional artificial neural network using fixed-length vectors to represent symbols has gained good performance in some specific tasks, it is still a black box that lacks interpretability, far from how humans perceive the world. Inspired by the grandmother-cell hypothesis in neuroscience, this work investigates how population encoding and spiking timing-dependent plasticity (STDP) mechanisms can be integrated into the learning of spiking neural networks, and how a population of neurons can represent a symbol via guiding the completion of sequential firing between different neuron populations. The neuron populations of different communities together constitute the entire commonsense knowledge graph, forming a giant graph spiking neural network. Moreover, we introduced the Reward-modulated spiking timing-dependent plasticity (R-STDP) mechanism to simulate the biological reinforcement learning process and completed the related reasoning tasks accordingly, achieving comparable accuracy and faster convergence speed than the graph convolutional artificial neural networks. For the fields of neuroscience and cognitive science, the work in this paper provided the foundation of computational modeling for further exploration of the way the human brain represents commonsense knowledge. For the field of artificial intelligence, this paper indicated the exploration direction for realizing a more robust and interpretable neural network by constructing a commonsense knowledge representation and reasoning spiking neural networks with solid biological plausibility.

preprint2022arXiv

Cosmo-Paleontology: Statistics of Fossil Groups in a Gravity-Only Simulation

We present a detailed study of fossil group candidates identified in "Last Journey", a gravity-only cosmological simulation covering a $(3.4\, h^{-1}\mathrm{Gpc})^3$ volume with a particle mass resolution of $m_p \approx 2.7 \times 10^9\, h^{-1}\mathrm{M}_\odot$. The simulation allows us to simultaneously capture a large number of group-scale halos and to resolve their internal structure. Historically, fossil groups have been characterized by high X-ray brightness and a large luminosity gap between the brightest and second brightest galaxy in the group. In order to identify candidate halos that host fossil groups, we use halo merger tree information to introduce two parameters: a luminous merger mass threshold ($M_\mathrm{LM}$) and a last luminous merger redshift cut-off ($z_\mathrm{LLM}$). The final parameter choices are informed by observational data and allow us to identify a plausible fossil group sample from the simulation. The candidate halos are characterized by reduced substructure and are therefore less likely to host bright galaxies beyond the brightest central galaxy. We carry out detailed studies of this sample, including analysis of halo properties and clustering. We find that our simple assumptions lead to fossil group candidates that form early, have higher concentrations, and are more relaxed compared to other halos in the same mass range.

preprint2022arXiv

Cross-Silo Federated Learning: Challenges and Opportunities

Federated learning (FL) is an emerging technology that enables the training of machine learning models from multiple clients while keeping the data distributed and private. Based on the participating clients and the model training scale, federated learning can be classified into two types: cross-device FL where clients are typically mobile devices and the client number can reach up to a scale of millions; cross-silo FL where clients are organizations or companies and the client number is usually small (e.g., within a hundred). While existing studies mainly focus on cross-device FL, this paper aims to provide an overview of the cross-silo FL. More specifically, we first discuss applications of cross-silo FL and outline its major challenges. We then provide a systematic overview of the existing approaches to the challenges in cross-silo FL by focusing on their connections and differences to cross-device FL. Finally, we discuss future directions and open issues that merit research efforts from the community.

preprint2022arXiv

Design of wavelength division multiplexing devices based on tunable edge states of valley photonic crystals

Wavelength division multiplexing (WDM) devices are key elements of Photonic integrated circuits (PICs). Conventional WDM devices based on silicon waveguides and photonic crystals have limited transmittance due to high loss introduced by the strong backward scattering from defects. In addition, it is challenging to reduce the footprint of those devices. Here we theoretically demonstrate a WDM device in the telecommunication range based on all-dielectric silicon topological valley photonic crystal (VPC) structures. We tune its effective refractive index by tuning the physical parameters of the lattice in the silicon substrate, which can continuously tune the working wavelength range of the topological edge states, which allows designing WDM devices with different channels. The WDM device has two channels (1470 nm-1523 nm and 1548 nm-1609 nm), with contrast ratios of 22.4 dB and 24.9 dB, respectively. The principle of manipulating the working bandwidth of the topological edge states can be generally applied in designing different integratable photonic devices, thus it will find broad applications.

preprint2022arXiv

Dwarf AGNs from Optical Variability for the Origins of Seeds (DAVOS): Insights from the Dark Energy Survey Deep Fields

We present a sample of 706, $z < 1.5$ active galactic nuclei (AGNs) selected from optical photometric variability in three of the Dark Energy Survey (DES) deep fields (E2, C3, and X3) over an area of 4.64 deg$^2$. We construct light curves using difference imaging aperture photometry for resolved sources and non-difference imaging PSF photometry for unresolved sources, respectively, and characterize the variability significance. Our DES light curves have a mean cadence of 7 days, a 6 year baseline, and a single-epoch imaging depth of up to $g \sim 24.5$. Using spectral energy distribution (SED) fitting, we find 26 out of total 706 variable galaxies are consistent with dwarf galaxies with a reliable stellar mass estimate ($M_{\ast}<10^{9.5}\ M_\odot$; median photometric redshift of 0.9). We were able to constrain rapid characteristic variability timescales ($\sim$ weeks) using the DES light curves in 15 dwarf AGN candidates (a subset of our variable AGN candidates) at a median photometric redshift of 0.4. This rapid variability is consistent with their low black hole masses. We confirm the low-mass AGN nature of one source with a high S/N optical spectrum. We publish our catalog, optical light curves, and supplementary data, such as X-ray properties and optical spectra, when available. We measure a variable AGN fraction versus stellar mass and compare to results from a forward model. This work demonstrates the feasibility of optical variability to identify AGNs with lower black hole masses in deep fields, which may be more &#34;pristine&#34; analogs of supermassive black hole seeds.

preprint2022arXiv

Enhancing Local Feature Learning for 3D Point Cloud Processing using Unary-Pairwise Attention

We present a simple but effective attention named the unary-pairwise attention (UPA) for modeling the relationship between 3D point clouds. Our idea is motivated by the analysis that the standard self-attention (SA) that operates globally tends to produce almost the same attention maps for different query positions, revealing difficulties for learning query-independent and query-dependent information jointly. Therefore, we reformulate the SA and propose query-independent (Unary) and query-dependent (Pairwise) components to facilitate the learning of both terms. In contrast to the SA, the UPA ensures query dependence via operating locally. Extensive experiments show that the UPA outperforms the SA consistently on various point cloud understanding tasks including shape classification, part segmentation, and scene segmentation. Moreover, simply equipping the popular PointNet++ method with the UPA even outperforms or is on par with the state-of-the-art attention-based approaches. In addition, the UPA systematically boosts the performance of both standard and modern networks when it is integrated into them as a compositional module.

preprint2022arXiv

Enhancing Local Feature Learning Using Diffusion for 3D Point Cloud Understanding

Learning point clouds is challenging due to the lack of connectivity information, i.e., edges. Although existing edge-aware methods can improve the performance by modeling edges, how edges contribute to the improvement is unclear. In this study, we propose a method that automatically learns to enhance/suppress edges while keeping the its working mechanism clear. First, we theoretically figure out how edge enhancement/suppression works. Second, we experimentally verify the edge enhancement/suppression behavior. Third, we empirically show that this behavior improves performance. In general, we observe that the proposed method achieves competitive performance in point cloud classification and segmentation tasks.

preprint2022arXiv

Enhancing Local Geometry Learning for 3D Point Cloud via Decoupling Convolution

Modeling the local surface geometry is challenging in 3D point cloud understanding due to the lack of connectivity information. Most prior works model local geometry using various convolution operations. We observe that the convolution can be equivalently decomposed as a weighted combination of a local and a global component. With this observation, we explicitly decouple these two components so that the local one can be enhanced and facilitate the learning of local surface geometry. Specifically, we propose Laplacian Unit (LU), a simple yet effective architectural unit that can enhance the learning of local geometry. Extensive experiments demonstrate that networks equipped with LUs achieve competitive or superior performance on typical point cloud understanding tasks. Moreover, through establishing connections between the mean curvature flow, a further investigation of LU based on curvatures is made to interpret the adaptive smoothing and sharpening effect of LU. The code will be available.

preprint2022arXiv

Ethereum Fraud Detection with Heterogeneous Graph Neural Networks

While transactions with cryptocurrencies such as Ethereum are becoming more prevalent, fraud and other criminal transactions are not uncommon. Graph analysis algorithms and machine learning techniques detect suspicious transactions that lead to phishing in large transaction networks. Many graph neural network (GNN) models have been proposed to apply deep learning techniques to graph structures. Although there is research on phishing detection using GNN models in the Ethereum transaction network, models that address the scale of the number of vertices and edges and the imbalance of labels have not yet been studied. In this paper, we compared the model performance of GNN models on the actual Ethereum transaction network dataset and phishing reported label data to exhaustively compare and verify which GNN models and hyperparameters produce the best accuracy. Specifically, we evaluated the model performance of representative homogeneous GNN models which consider single-type nodes and edges and heterogeneous GNN models which support different types of nodes and edges. We showed that heterogeneous models had better model performance than homogeneous models. In particular, the RGCN model achieved the best performance in the overall metrics.

preprint2022arXiv

Exploration, Exploitation, and Engagement in Multi-Armed Bandits with Abandonment

Multi-armed bandit (MAB) is a classic model for understanding the exploration-exploitation trade-off. The traditional MAB model for recommendation systems assumes the user stays in the system for the entire learning horizon. In new online education platforms such as ALEKS or new video recommendation systems such as TikTok and YouTube Shorts, the amount of time a user spends on the app depends on how engaging the recommended contents are. Users may temporarily leave the system if the recommended items cannot engage the users. To understand the exploration, exploitation, and engagement in these systems, we propose a new model, called MAB-A where &#34;A&#34; stands for abandonment and the abandonment probability depends on the current recommended item and the user&#39;s past experience (called state). We propose two algorithms, ULCB and KL-ULCB, both of which do more exploration (being optimistic) when the user likes the previous recommended item and less exploration (being pessimistic) when the user does not like the previous item. We prove that both ULCB and KL-ULCB achieve logarithmic regret, $O(\log K)$, where $K$ is the number of visits (or episodes). Furthermore, the regret bound under KL-ULCB is asymptotically sharp. We also extend the proposed algorithms to the general-state setting. Simulation results confirm our theoretical analysis and show that the proposed algorithms have significantly lower regrets than the traditional UCB and KL-UCB, and Q-learning-based algorithms.

preprint2022arXiv

Extremal trees of given degree sequence or segment sequence with respect to Steiner 3-eccentricity

The Steiner $k$-eccentricity of a vertex in graph $G$ is the maximum Steiner distance over all $k$-subsets containing the vertex. %Some general properties of the Steiner 3-eccentricity of trees are given. Let $\mathbb{T}_n$ be the set of all $n$-vertex trees, $\mathbb{T}_{n,Δ}$ be the set of $n$-vertex trees with given maximum degree equal to $Δ$, $\mathbb{T}_n^k$ be the set of $n$-vertex trees with exactly $k$ vertices of maximum degree, and let $\mathbb{T}_{n,Δ}^k$ be the set of $n$-vertex trees with exactly $k$ vertices of given maximum degree equal to $Δ.$ In this paper, we first determine the sharp upper bound on the average Steiner 3-eccentricity of $n$-vertex trees with given degree sequence. The corresponding extremal graph is characterized. Consequently, together with majorization theory, the unique graph among $\mathbb{T}_n$ (resp. $\mathbb{T}_{n,Δ}$, $\mathbb{T}_n^k, \mathbb{T}_{n,Δ}^k$) having the maximum average Steiner 3-eccentricity is identified. Then we characterize the unique $n$-vertex tree with given segment sequence having the largest average Steiner 3-eccentricity. Similarly, the $n$-vertex tree with given number of segments having the largest average Steiner 3-eccentricity is determined.

preprint2022arXiv

FastAdaBelief: Improving Convergence Rate for Belief-based Adaptive Optimizers by Exploiting Strong Convexity

AdaBelief, one of the current best optimizers, demonstrates superior generalization ability compared to the popular Adam algorithm by viewing the exponential moving average of observed gradients. AdaBelief is theoretically appealing in that it has a data-dependent $O(\sqrt{T})$ regret bound when objective functions are convex, where $T$ is a time horizon. It remains however an open problem whether the convergence rate can be further improved without sacrificing its generalization ability. %on how to exploit strong convexity to further improve the convergence rate of AdaBelief. To this end, we make a first attempt in this work and design a novel optimization algorithm called FastAdaBelief that aims to exploit its strong convexity in order to achieve an even faster convergence rate. In particular, by adjusting the step size that better considers strong convexity and prevents fluctuation, our proposed FastAdaBelief demonstrates excellent generalization ability as well as superior convergence. As an important theoretical contribution, we prove that FastAdaBelief attains a data-dependant $O(\log T)$ regret bound, which is substantially lower than AdaBelief. On the empirical side, we validate our theoretical analysis with extensive experiments in both scenarios of strong and non-strong convexity on three popular baseline models. Experimental results are very encouraging: FastAdaBelief converges the quickest in comparison to all mainstream algorithms while maintaining an excellent generalization ability, in cases of both strong or non-strong convexity. FastAdaBelief is thus posited as a new benchmark model for the research community.

preprint2022arXiv

Federated Remote Physiological Measurement with Imperfect Data

The growing need for technology that supports remote healthcare is being acutely highlighted by an aging population and the COVID-19 pandemic. In health-related machine learning applications the ability to learn predictive models without data leaving a private device is attractive, especially when these data might contain features (e.g., photographs or videos of the body) that make identifying a subject trivial and/or the training data volume is large (e.g., uncompressed video). Camera-based remote physiological sensing facilitates scalable and low-cost measurement, but is a prime example of a task that involves analysing high bit-rate videos containing identifiable images and sensitive health information. Federated learning enables privacy-preserving decentralized training which has several properties beneficial for camera-based sensing. We develop the first mobile federated learning camera-based sensing system and show that it can perform competitively with traditional state-of-the-art supervised approaches. However, in the presence of corrupted data (e.g., video or label noise) from a few devices the performance of weight averaging quickly degrades. To address this, we leverage knowledge about the expected noise profile within the video to intelligently adjust how the model weights are averaged on the server. Our results show that this significantly improves upon the robustness of models even when the signal-to-noise ratio is low

preprint2022arXiv

FedSSO: A Federated Server-Side Second-Order Optimization Algorithm

In this work, we propose FedSSO, a server-side second-order optimization method for federated learning (FL). In contrast to previous works in this direction, we employ a server-side approximation for the Quasi-Newton method without requiring any training data from the clients. In this way, we not only shift the computation burden from clients to server, but also eliminate the additional communication for second-order updates between clients and server entirely. We provide theoretical guarantee for convergence of our novel method, and empirically demonstrate our fast convergence and communication savings in both convex and non-convex settings.

preprint2022arXiv

GNNSampler: Bridging the Gap between Sampling Algorithms of GNN and Hardware

Sampling is a critical operation in Graph Neural Network (GNN) training that helps reduce the cost. Previous literature has explored improving sampling algorithms via mathematical and statistical methods. However, there is a gap between sampling algorithms and hardware. Without consideration of hardware, algorithm designers merely optimize sampling at the algorithm level, missing the great potential of promoting the efficiency of existing sampling algorithms by leveraging hardware features. In this paper, we pioneer to propose a unified programming model for mainstream sampling algorithms, termed GNNSampler, covering the critical processes of sampling algorithms in various categories. Second, to leverage the hardware feature, we choose the data locality as a case study, and explore the data locality among nodes and their neighbors in a graph to alleviate irregular memory access in sampling. Third, we implement locality-aware optimizations in GNNSampler for various sampling algorithms to optimize the general sampling process. Finally, we emphatically conduct experiments on large graph datasets to analyze the relevance among training time, accuracy, and hardware-level metrics. Extensive experiments show that our method is universal to mainstream sampling algorithms and helps significantly reduce the training time, especially in large-scale graphs.

preprint2022arXiv

Incoporating Weighted Board Learning System for Accurate Occupational Pneumoconiosis Staging

Occupational pneumoconiosis (OP) staging is a vital task concerning the lung healthy of a subject. The staging result of a patient is depended on the staging standard and his chest X-ray. It is essentially an image classification task. However, the distribution of OP data is commonly imbalanced, which largely reduces the effect of classification models which are proposed under the assumption that data follow a balanced distribution and causes inaccurate staging results. To achieve accurate OP staging, we proposed an OP staging model who is able to handle imbalance data in this work. The proposed model adopts gray level co-occurrence matrix (GLCM) to extract texture feature of chest X-ray and implements classification with a weighted broad learning system (WBLS). Empirical studies on six data cases provided by a hospital indicate that proposed model can perform better OP staging than state-of-the-art classifiers with imbalanced data.

preprint2022arXiv

Large-System Insensitivity of Zero-Waiting Load Balancing Algorithms

This paper studies the sensitivity (or insensitivity) of a class of load balancing algorithms that achieve asymptotic zero-waiting in the sub-Halfin-Whitt regime, named LB-zero. Most existing results on zero-waiting load balancing algorithms assume the service time distribution is exponential. This paper establishes the {\em large-system insensitivity} of LB-zero for jobs whose service time follows a Coxian distribution with a finite number of phases. This result suggests that LB-zero achieves asymptotic zero-waiting for a large class of service time distributions, which is confirmed in our simulations. To prove this result, this paper develops a new technique, called &#34;Iterative State-Space Peeling&#34; (or ISSP for short). ISSP first identifies an iterative relation between the upper and lower bounds on the queue states and then proves that the system lives near the fixed point of the iterative bounds with a high probability. Based on ISSP, the steady-state distribution of the system is further analyzed by applying Stein&#39;s method in the neighborhood of the fixed point. ISSP, like state-space collapse in heavy-traffic analysis, is a general approach that may be used to study other complex stochastic systems.

preprint2022arXiv

Learning Higher-Order Dynamics in Video-Based Cardiac Measurement

Computer vision methods typically optimize for first-order dynamics (e.g., optical flow). However, in many cases the properties of interest are subtle variations in higher-order changes, such as acceleration. This is true in the cardiac pulse, where the second derivative can be used as an indicator of blood pressure and arterial disease. Recent developments in camera-based vital sign measurement have shown that cardiac measurements can be recovered with impressive accuracy from videos; however, most of the research has focused on extracting summary statistics such as heart rate. Less emphasis has been put on the accuracy of waveform morphology that is necessary for many clinically meaningful assessments. In this work, we provide evidence that higher-order dynamics are better estimated by neural models when explicitly optimized for in the loss function. Furthermore, adding second-derivative inputs also improves performance when estimating second-order dynamics. We illustrate this, by showing that incorporating the second derivative of both the input frames and the target vital sign signals into the training procedure, models are better able to estimate left ventricle ejection time (LVET) intervals.

preprint2022arXiv

Linearly-constrained nonsmooth optimization for training autoencoders

A regularized minimization model with $l_1$-norm penalty (RP) is introduced for training the autoencoders that belong to a class of two-layer neural networks. We show that the RP can act as an exact penalty model which shares the same global minimizers, local minimizers, and d(irectional)-stationary points with the original regularized model under mild conditions. We construct a bounded box region that contains at least one global minimizer of the RP, and propose a linearly constrained regularized minimization model with $l_1$-norm penalty (LRP) for training autoencoders. A smoothing proximal gradient algorithm is designed to solve the LRP. Convergence of the algorithm to a generalized d-stationary point of the RP and LRP is delivered. Comprehensive numerical experiments convincingly illustrate the efficiency as well as the robustness of the proposed algorithm.

preprint2022arXiv

MMChat: Multi-Modal Chat Dataset on Social Media

Incorporating multi-modal contexts in conversation is important for developing more engaging dialogue systems. In this work, we explore this direction by introducing MMChat: a large-scale Chinese multi-modal dialogue corpus (32.4M raw dialogues and 120.84K filtered dialogues). Unlike previous corpora that are crowd-sourced or collected from fictitious movies, MMChat contains image-grounded dialogues collected from real conversations on social media, in which the sparsity issue is observed. Specifically, image-initiated dialogues in common communications may deviate to some non-image-grounded topics as the conversation proceeds. To better investigate this issue, we manually annotate 100K dialogues from MMChat and further filter the corpus accordingly, which yields MMChat-hf. We develop a benchmark model to address the sparsity issue in dialogue generation tasks by adapting the attention routing mechanism on image features. Experiments demonstrate the usefulness of incorporating image features and the effectiveness of handling the sparsity of image features.

preprint2022arXiv

MobilePhys: Personalized Mobile Camera-Based Contactless Physiological Sensing

Camera-based contactless photoplethysmography refers to a set of popular techniques for contactless physiological measurement. The current state-of-the-art neural models are typically trained in a supervised manner using videos accompanied by gold standard physiological measurements. However, they often generalize poorly out-of-domain examples (i.e., videos that are unlike those in the training set). Personalizing models can help improve model generalizability, but many personalization techniques still require some gold standard data. To help alleviate this dependency, in this paper, we present a novel mobile sensing system called MobilePhys, the first mobile personalized remote physiological sensing system, that leverages both front and rear cameras on a smartphone to generate high-quality self-supervised labels for training personalized contactless camera-based PPG models. To evaluate the robustness of MobilePhys, we conducted a user study with 39 participants who completed a set of tasks under different mobile devices, lighting conditions/intensities, motion tasks, and skin types. Our results show that MobilePhys significantly outperforms the state-of-the-art on-device supervised training and few-shot adaptation methods. Through extensive user studies, we further examine how does MobilePhys perform in complex real-world settings. We envision that calibrated or personalized camera-based contactless PPG models generated from our proposed dual-camera mobile sensing system will open the door for numerous future applications such as smart mirrors, fitness and mobile health applications.

preprint2022arXiv

NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video: Dataset, Methods and Results

This paper reviews the NTIRE 2022 Challenge on Super-Resolution and Quality Enhancement of Compressed Video. In this challenge, we proposed the LDV 2.0 dataset, which includes the LDV dataset (240 videos) and 95 additional videos. This challenge includes three tracks. Track 1 aims at enhancing the videos compressed by HEVC at a fixed QP. Track 2 and Track 3 target both the super-resolution and quality enhancement of HEVC compressed video. They require x2 and x4 super-resolution, respectively. The three tracks totally attract more than 600 registrations. In the test phase, 8 teams, 8 teams and 12 teams submitted the final results to Tracks 1, 2 and 3, respectively. The proposed methods and solutions gauge the state-of-the-art of super-resolution and quality enhancement of compressed video. The proposed LDV 2.0 dataset is available at https://github.com/RenYang-home/LDV_dataset. The homepage of this challenge (including open-sourced codes) is at https://github.com/RenYang-home/NTIRE22_VEnh_SR.

preprint2022arXiv

Numerical study of PbTe-Pb hybrid nanowires for engineering Majorana zero modes

Epitaxial semiconductor-superconductor (SM-SC) hybrid nanowires are potential candidates for implementing Majorana qubits. Recent experimental and theoretical works show that charged impurities in SM remain a major problem in all existing hybrid nanowires, in which the SM is either InAs or InSb while the SC is mainly Al. Here, we theoretically validate the recently proposed PbTe-Pb hybrid nanowire as a potential candidate for Majorana devices. By studying the electrostatic and electronic properties of PbTe nanowires, we demonstrate that the huge dielectric constant of PbTe endows itself a high tolerance of charged impurity, which is a potential advantage over InAs and InSb nanowires. Moreover, we find that the effective axial Landé $g$ factor and Rashba spin-orbit coupling strength of PbTe nanowires are comparable to those of InAs nanowires. The conceivable merits of using Pb as the SC are (i) Pb has a larger superconducting gap, higher critical temperature, and higher parallel critical magnetic field than those of Al; (ii) a superconducting gap comparable with those of InAs-Al and InSb-Al can be induced in PbTe-Pb even by a weak coupling between Pb and PbTe, which simultaneously relieves the adverse renormalization and induced disorder effects on SM from SC; and (iii) Pb film can be grown on PbTe with a thin buffer CdTe layer in between, solving the lattice mismatch problem as an important source of disorder. In the presence of a parallel magnetic field, we show that the typical BdG energy spectrum and tunneling spectroscopy of PbTe-Pb resemble those of InAs and InSb based hybrid nanowires exposed to a tilting magnetic field, as a result of the highly anisotropic Landé $g$ factors of PbTe nanowires. The calculated topological phase diagrams of PbTe-Pb indicate that the multivalley character of PbTe makes it easier than InAs and InSb to access topological superconducting phases.

preprint2022arXiv

On the effect of fast rotation and vertical viscosity on the lifespan of the $3D$ primitive equations

We study the effect of the fast rotation and vertical viscosity on the lifespan of solutions to the three-dimensional primitive equations (also known as the hydrostatic Navier-Stokes equations) with impermeable and stress-free boundary conditions. Firstly, for a short time interval, independent of the rate of rotation $|Ω|$, we establish the local well-posedness of solutions with initial data that is analytic in the horizontal variables and only $L^2$ in the vertical variable. Moreover, it is shown that the solutions immediately become analytic in all the variables with increasing-in-time (at least linearly) radius of analyticity in the vertical variable for as long as the solutions exist. On the other hand, the radius of analyticity in the horizontal variables might decrease with time, but as long as it remains positive the solution exists. Secondly, with fast rotation, i.e., large $|Ω|$, we show that the existence time of the solution can be prolonged, with &#34;well-prepared&#34; initial data. Finally, in the case of two spatial dimensions with $Ω=0$, we establish the global well-posedness provided that the initial data is small enough. The smallness condition on the initial data depends on the vertical viscosity and the initial radius of analyticity in the horizontal variables.

preprint2022arXiv

On the resolvability of the dynamics of one fluid flow via a testing fluid in a two-fluids flow model

In this work, by considering an isentropic fluid-fluid interaction model with a large symmetric drag force, a commonly used simplified two-fluids flow model is justified as the asymptotic limit. Equations for each fluid component with an interaction term are identified in addition to the simplified two-fluids flow model, which can be used to resolve the density of one fluid specie based on information on density and the velocity of the other fluid specie, i.e., the testing flow.

preprint2022arXiv

Optical Variability of Quasars with 20-Year Photometric Light Curves

We study the optical $gri$ photometric variability of a sample of 190 quasars within the SDSS Stripe 82 region that have long-term photometric coverage during $\sim 1998-2020$ with SDSS, PanSTARRS-1, the Dark Energy Survey, and dedicated follow-up monitoring with Blanco 4m/DECam. With on average $\sim 200$ nightly epochs per quasar per filter band, we improve the parameter constraints from a Damped Random Walk (DRW) model fit to the light curves over previous studies with 10-15 yr baselines and $\lesssim 100$ epochs. We find that the average damping timescale $τ_{\rm DRW}$ continues to rise with increased baseline, reaching a median value of $\sim 750$ days ($g$ band) in the rest-frame of these quasars using the 20-yr light curves. Some quasars may have gradual, long-term trends in their light curves, suggesting that either the DRW fit requires very long baselines to converge, or that the underlying variability is more complex than a single DRW process for these quasars. Using a subset of quasars with better-constrained $τ_{\rm DRW}$ (less than 20\% of the baseline), we confirm a weak wavelength dependence of $τ_{\rm DRW}\propto λ^{0.51\pm0.20}$. We further quantify optical variability of these quasars over days to decades timescales using structure function (SF) and power spectrum density (PSD) analyses. The SF and PSD measurements qualitatively confirm the measured (hundreds of days) damping timescales from the DRW fits. However, the ensemble PSD is steeper than that of a DRW on timescales less than $\sim$ a month for these luminous quasars, and this second break point correlates with the longer DRW damping timescale.

preprint2022arXiv

Perturbative QCD analysis of neutral $B$-meson decays into $σσ, σf_0$ and $f_0 f_0$

The decays of $B_{d,s}^0 \to σσ, σf_0, f_0 f_0$, with $σ$ and $f_0$ denoting the light scalar mesons $f_0(500)$ and $f_0(980)$ in the two-quark picture, are studied in the perturbative QCD approach based on $k_T$ factorization. With the referenced value of the mixing angle $|φ| \sim 25^\circ$ for the $σ-f_0$ mixing in the quark-flavor basis, it is of great interest to obtain that: (a) these neutral $B$-meson decays into $σσ, σf_0$, and $f_0 f_0$ have large branching ratios in the order of $10^{-6} \sim 10^{-4}$, which mean the possibly constructive interferences existed in the decays with different flavor states, and then are expected to be tested at the Large Hadron Collider beauty and/or Belle-II experiments in the (near) future; (b) the large direct CP violations could be easily found in the $B_d^0 \to σσ, f_0 f_0$ and $B_{d,s}^0 \to σf_0$ decays, which indicate the considerable interferences between the tree and the penguin decay amplitudes involved in these four modes, and would be confronted with the future measurements; (c) these neutral $B$-meson decays could be examined through the secondary decay chain $σ/f_0 \to π^+ π^-$, namely, the four-body decays of $B_{d,s}^0 \to (π^+ π^-)_{σ(f_0)} (π^+ π^-)_{σ(f_0)}$. On the other side, it seems that other 4 four-body decays of $B_d^0 \to (π^+ π^-)_σ (K^+ K^-)_{f_0}$, $B_s^0 \to (π^+ π^-)_{σ(f_0)} (K^+ K^-)_{f_0}$, and $B_s^0 \to (K^+ K^-)_{f_0} (K^+ K^-)_{f_0}$ could also be detected at the relevant experiments, if the $f_0 \to K^+ K^-$ could be identified from the $ϕ\to K^+ K^-$ clearly.

preprint2022arXiv

Provable Convergence of Nesterov&#39;s Accelerated Gradient Method for Over-Parameterized Neural Networks

Momentum methods, such as heavy ball method~(HB) and Nesterov&#39;s accelerated gradient method~(NAG), have been widely used in training neural networks by incorporating the history of gradients into the current updating process. In practice, they often provide improved performance over (stochastic) gradient descent~(GD) with faster convergence. Despite these empirical successes, theoretical understandings of their accelerated convergence rates are still lacking. Recently, some attempts have been made by analyzing the trajectories of gradient-based methods in an over-parameterized regime, where the number of the parameters is significantly larger than the number of the training instances. However, the majority of existing theoretical work is mainly concerned with GD and the established convergence result of NAG is inferior to HB and GD, which fails to explain the practical success of NAG. In this paper, we take a step towards closing this gap by analyzing NAG in training a randomly initialized over-parameterized two-layer fully connected neural network with ReLU activation. Despite the fact that the objective function is non-convex and non-smooth, we show that NAG converges to a global minimum at a non-asymptotic linear rate $(1-Θ(1/\sqrtκ))^t$, where $κ> 1$ is the condition number of a gram matrix and $t$ is the number of the iterations. Compared to the convergence rate $(1-Θ(1/κ))^t$ of GD, our result provides theoretical guarantees for the acceleration of NAG in neural network training. Furthermore, our findings suggest that NAG and HB have similar convergence rate. Finally, we conduct extensive experiments on six benchmark datasets to validate the correctness of our theoretical results.

preprint2022arXiv

Rethinking Efficiency and Redundancy in Training Large-scale Graphs

Large-scale graphs are ubiquitous in real-world scenarios and can be trained by Graph Neural Networks (GNNs) to generate representation for downstream tasks. Given the abundant information and complex topology of a large-scale graph, we argue that redundancy exists in such graphs and will degrade the training efficiency. Unfortunately, the model scalability severely restricts the efficiency of training large-scale graphs via vanilla GNNs. Despite recent advances in sampling-based training methods, sampling-based GNNs generally overlook the redundancy issue. It still takes intolerable time to train these models on large-scale graphs. Thereby, we propose to drop redundancy and improve efficiency of training large-scale graphs with GNNs, by rethinking the inherent characteristics in a graph. In this paper, we pioneer to propose a once-for-all method, termed DropReef, to drop the redundancy in large-scale graphs. Specifically, we first conduct preliminary experiments to explore potential redundancy in large-scale graphs. Next, we present a metric to quantify the neighbor heterophily of all nodes in a graph. Based on both experimental and theoretical analysis, we reveal the redundancy in a large-scale graph, i.e., nodes with high neighbor heterophily and a great number of neighbors. Then, we propose DropReef to detect and drop the redundancy in large-scale graphs once and for all, helping reduce the training time while ensuring no sacrifice in the model accuracy. To demonstrate the effectiveness of DropReef, we apply it to recent state-of-the-art sampling-based GNNs for training large-scale graphs, owing to the high precision of such models. With DropReef leveraged, the training efficiency of models can be greatly promoted. DropReef is highly compatible and is offline performed, benefiting the state-of-the-art sampling-based GNNs in the present and future to a significant extent.

preprint2022arXiv

SCAMPS: Synthetics for Camera Measurement of Physiological Signals

The use of cameras and computational algorithms for noninvasive, low-cost and scalable measurement of physiological (e.g., cardiac and pulmonary) vital signs is very attractive. However, diverse data representing a range of environments, body motions, illumination conditions and physiological states is laborious, time consuming and expensive to obtain. Synthetic data have proven a valuable tool in several areas of machine learning, yet are not widely available for camera measurement of physiological states. Synthetic data offer &#34;perfect&#34; labels (e.g., without noise and with precise synchronization), labels that may not be possible to obtain otherwise (e.g., precise pixel level segmentation maps) and provide a high degree of control over variation and diversity in the dataset. We present SCAMPS, a dataset of synthetics containing 2,800 videos (1.68M frames) with aligned cardiac and respiratory signals and facial action intensities. The RGB frames are provided alongside segmentation maps. We provide precise descriptive statistics about the underlying waveforms, including inter-beat interval, heart rate variability, and pulse arrival time. Finally, we present baseline results training on these synthetic data and testing on real-world datasets to illustrate generalizability.

preprint2022arXiv

Selective clustering ensemble based on kappa and F-score

Clustering ensemble has an impressive performance in improving the accuracy and robustness of partition results and has received much attention in recent years. Selective clustering ensemble (SCE) can further improve the ensemble performance by selecting base partitions or clusters in according to diversity and stability. However, there is a conflict between diversity and stability, and how to make the trade-off between the two is challenging. The key here is how to evaluate the quality of the base partitions and clusters. In this paper, we propose a new evaluation method for partitions and clusters using kappa and F-score, leading to a new SCE method, which uses kappa to select informative base partitions and uses F-score to weight clusters based on stability. The effectiveness and efficiency of the proposed method is empirically validated over real datasets.

preprint2022arXiv

Spectroscopy Approaches for Food Safety Applications: Improving Data Efficiency Using Active Learning and Semi-Supervised Learning

The past decade witnesses a rapid development in the measurement and monitoring technologies for food science. Among these technologies, spectroscopy has been widely used for the analysis of food quality, safety, and nutritional properties. Due to the complexity of food systems and the lack of comprehensive predictive models, rapid and simple measurements to predict complex properties in food systems are largely missing. Machine Learning (ML) has shown great potential to improve classification and prediction of these properties. However, the barriers to collect large datasets for ML applications still persists. In this paper, we explore different approaches of data annotation and model training to improve data efficiency for ML applications. Specifically, we leverage Active Learning (AL) and Semi-Supervised Learning (SSL) and investigate four approaches: baseline passive learning, AL, SSL, and a hybrid of AL and SSL. To evaluate these approaches, we collect two spectroscopy datasets: predicting plasma dosage and detecting foodborne pathogen. Our experimental results show that, compared to the de facto passive learning approach, AL and SSL methods reduce the number of labeled samples by 50% and 25% for each ML application, respectively.

preprint2022arXiv

Survey on Graph Neural Network Acceleration: An Algorithmic Perspective

Graph neural networks (GNNs) have been a hot spot of recent research and are widely utilized in diverse applications. However, with the use of huger data and deeper models, an urgent demand is unsurprisingly made to accelerate GNNs for more efficient execution. In this paper, we provide a comprehensive survey on acceleration methods for GNNs from an algorithmic perspective. We first present a new taxonomy to classify existing acceleration methods into five categories. Based on the classification, we systematically discuss these methods and highlight their correlations. Next, we provide comparisons from aspects of the efficiency and characteristics of these methods. Finally, we suggest some promising prospects for future research.

preprint2022arXiv

The soundproof model of an acoustic--internal waves system with low stratification

This work is devoted to investigating a compressible fluid system with low stratification, which is driven by fast acoustic waves and internal waves. The approximation using a soundproof model is justified. More precisely, the soundproof model captures the dynamics of both the non-oscillating mean flows and the oscillating internal waves, while filters out the fast acoustic waves, of the compressible system with or without initial acoustic waves. Moreover, the fast-slow oscillation structure is investigated.

preprint2022arXiv

User-Level Membership Inference Attack against Metric Embedding Learning

Membership inference (MI) determines if a sample was part of a victim model training set. Recent development of MI attacks focus on record-level membership inference which limits their application in many real-world scenarios. For example, in the person re-identification task, the attacker (or investigator) is interested in determining if a user&#39;s images have been used during training or not. However, the exact training images might not be accessible to the attacker. In this paper, we develop a user-level MI attack where the goal is to find if any sample from the target user has been used during training even when no exact training sample is available to the attacker. We focus on metric embedding learning due to its dominance in person re-identification, where user-level MI attack is more sensible. We conduct an extensive evaluation on several datasets and show that our approach achieves high accuracy on user-level MI task.

preprint2022arXiv

Very Large Array Multi-band Radio Imaging of the Triple AGN Candidate SDSS J0849+1114

Kpc-scale triple active galactic nuclei (AGNs), potential precursors of gravitationally-bound triple massive black holes (MBHs), are rarely seen objects and believed to play an important role in the evolution of MBHs and their host galaxies. In this work we present a multi-band (3.0, 6.0 10.0, and 15.0 GHz), high-resolution radio imaging of the triple AGN candidate, SDSS J0849+1114, using the Very Large Array. Two of the three nuclei (A and C) are detected at 3.0, 6.0, and 15 GHz for the first time, both exhibiting a steep spectrum over 3--15 GHz (with a spectral index $-0.90 \pm 0.05$ and $-1.03 \pm 0.04$) consistent with a synchrotron origin. Nucleus A, the strongest nucleus among the three, shows a double-sided jet, with the jet orientation changing by $\sim20^{\circ}$ between its inner 1&#34; and the outer 5.5&#34; (8.1 kpc) components, which may be explained as the MBH&#39;s angular momentum having been altered by merger-enhanced accretion. Nucleus C also shows a two-sided jet, with the western jet inflating into a radio lobe with an extent of 1.5&#34; (2.2 kpc). The internal energy of the radio lobe is estimated to be $\rm 5.0 \times 10^{55}$ erg, for an equipartition magnetic field strength of $\rm \sim 160\ μG$. No significant radio emission is detected at all four frequencies for nucleus B, yielding an upper limit of 15, 15, 15, and 18 $\rm μJy\ beam^{-1}$ at 3.0, 6.0, 10.0, and 15.0 GHz, based on which we constrain the star formation rate in nucleus B to be $\lesssim 0.4~\rm M_{\odot}~yr^{-1}$.

preprint2022arXiv

Well-posedness of Hibler&#39;s dynamical sea-ice model

This paper establishes the local-in-time well-posedness of solutions to an approximating system constructed by mildly regularizing the dynamical sea-ice model of {\it W.D. Hibler, Journal of Physical Oceanography, 1979}. Our choice of regularization has been carefully designed, prompted by physical considerations, to retain the original coupled hyperbolic-parabolic character of Hibler&#39;s model. Various regularized versions of this model have been used widely for the numerical simulation of the circulation and thickness of the Arctic ice cover. However, due to the singularity in the ice rheology, the notion of solutions to the original model is unclear. Instead, an approximating system, which captures current numerical study, is proposed. The well-posedness theory of such a system provides a first-step groundwork in both numerical study and future analytical study.

preprint2022arXiv

YouTube, The Great Radicalizer? Auditing and Mitigating Ideological Biases in YouTube Recommendations

Recommendations algorithms of social media platforms are often criticized for placing users in &#34;rabbit holes&#34; of (increasingly) ideologically biased content. Despite these concerns, prior evidence on this algorithmic radicalization is inconsistent. Furthermore, prior work lacks systematic interventions that reduce the potential ideological bias in recommendation algorithms. We conduct a systematic audit of YouTube&#39;s recommendation system using a hundred thousand sock puppets to determine the presence of ideological bias (i.e., are recommendations aligned with users&#39; ideology), its magnitude (i.e., are users recommended an increasing number of videos aligned with their ideology), and radicalization (i.e., are the recommendations progressively more extreme). Furthermore, we design and evaluate a bottom-up intervention to minimize ideological bias in recommendations without relying on cooperation from YouTube. We find that YouTube&#39;s recommendations do direct users -- especially right-leaning users -- to ideologically biased and increasingly radical content on both homepages and in up-next recommendations. Our intervention effectively mitigates the observed bias, leading to more recommendations to ideologically neutral, diverse, and dissimilar content, yet debiasing is especially challenging for right-leaning users. Our systematic assessment shows that while YouTube recommendations lead to ideological bias, such bias can be mitigated through our intervention.

preprint2022arXiv

Zero Mach Number Limit of the Compressible Primitive Equations: Ill-prepared Initial Data

In the work, we consider the zero Mach number limit of compressible primitive equations in the domain $\mathbb{R}^2 \times 2\mathbb{T}$ or $\mathbb{T}^2 \times 2\mathbb{T}$. We identify the limit equations to be the primitive equations with the incompressible condition. The convergence behaviors are studied in both $\mathbb{R}^2 \times 2\mathbb{T}$ and $\mathbb{T}^2 \times 2\mathbb{T}$, respectively. This paper takes into account the high oscillating acoustic waves and is an extension of our previous work by X. Liu and E.S. Titi, Arch. Rational Mech. Anal., 238, 705-747, 2020.

preprint2021arXiv

Automated Model Design and Benchmarking of 3D Deep Learning Models for COVID-19 Detection with Chest CT Scans

The COVID-19 pandemic has spread globally for several months. Because its transmissibility and high pathogenicity seriously threaten people&#39;s lives, it is crucial to accurately and quickly detect COVID-19 infection. Many recent studies have shown that deep learning (DL) based solutions can help detect COVID-19 based on chest CT scans. However, most existing work focuses on 2D datasets, which may result in low quality models as the real CT scans are 3D images. Besides, the reported results span a broad spectrum on different datasets with a relatively unfair comparison. In this paper, we first use three state-of-the-art 3D models (ResNet3D101, DenseNet3D121, and MC3\_18) to establish the baseline performance on the three publicly available chest CT scan datasets. Then we propose a differentiable neural architecture search (DNAS) framework to automatically search for the 3D DL models for 3D chest CT scans classification with the Gumbel Softmax technique to improve the searching efficiency. We further exploit the Class Activation Mapping (CAM) technique on our models to provide the interpretability of the results. The experimental results show that our automatically searched models (CovidNet3D) outperform the baseline human-designed models on the three datasets with tens of times smaller model size and higher accuracy. Furthermore, the results also verify that CAM can be well applied in CovidNet3D for COVID-19 datasets to provide interpretability for medical diagnosis.

preprint2021arXiv

Charmless $B_s\to V S$ Decays in PQCD Approach

In this work, we investigate the $B_s\to V S$ decays in the perturbative QCD approach, where $V$ and $S$ denote the vector meson and scalar meson respectively. Based on the two-quark structure, considering two different scenarios for describing the scalar mesons, we calculate the branching fractions and the direct $CP$ asymmetries of all $B_s\to VS$ decays. Most branching fractions are predicted to be at $10^{-7}$ to $10^{-5}$, which could be measured in the LHCb and Belle-II experiments, especially for these color-allowed $B_s\to κ(800)(K_0^*(1430))K^*$ decays. It is found that the branching fractions of $B_s\to K_0^{*0}(1430)\bar{K}^{*0}$ and $B_s\to K_0^{*+}(1430)\bar{K}^{*-}$ are very sensitive to the scenarios, which can be used to determine whether $K_0^{*0}(1430)$ belongs to the ground state or the first excited state, if the data were available. We also note that some decays have large direct $CP$ asymmetries, some of which are also sensitive to the scenarios, such as the $B_s \to a_0^+(1450)K^{*-}$ and the $B_s\to f_0(1500) K^{*0}$ decays. Since the experimental measurements of $B_s\to VS$ decays are on the way, combined with the available data in the future, we expect the theoretical predictions will shed light on the structure of the scalar mesons.

preprint2021arXiv

Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Function

In this paper, we focus on the decentralized optimization problem over the Stiefel manifold, which is defined on a connected network of $d$ agents. The objective is an average of $d$ local functions, and each function is privately held by an agent and encodes its data. The agents can only communicate with their neighbors in a collaborative effort to solve this problem. In existing methods, multiple rounds of communications are required to guarantee the convergence, giving rise to high communication costs. In contrast, this paper proposes a decentralized algorithm, called DESTINY, which only invokes a single round of communications per iteration. DESTINY combines gradient tracking techniques with a novel approximate augmented Lagrangian function. The global convergence to stationary points is rigorously established. Comprehensive numerical experiments demonstrate that DESTINY has a strong potential to deliver a cutting-edge performance in solving a variety of testing problems.

preprint2021arXiv

DST: Data Selection and joint Training for Learning with Noisy Labels

Training a deep neural network heavily relies on a large amount of training data with accurate annotations. To alleviate this problem, various methods have been proposed to annotate the data automatically. However, automatically generating annotations will inevitably yields noisy labels. In this paper, we propose a Data Selection and joint Training (DST) method to automatically select training samples with accurate annotations. Specifically, DST fits a mixture model according to the original annotation as well as the predicted label for each training sample, and the mixture model is utilized to dynamically divide the training dataset into a correctly labeled dataset, a correctly predicted set and a wrong dataset. Then, DST is trained with these datasets in a supervised manner. Due to confirmation bias problem, we train the two networks alternately, and each network is tasked to establish the data division to teach another network. For each iteration, the correctly labeled and predicted labels are reweighted respectively by the probabilities from the mixture model, and a uniform distribution is used to generate the probabilities of the wrong samples. Experiments on CIFAR-10, CIFAR-100 and Clothing1M demonstrate that DST is the comparable or superior to the state-of-the-art methods.

preprint2021arXiv

Investigating the Accretion Nature of Binary Supermassive Black Hole Candidate SDSS J025214.67-002813.7

We present results on a multi-wavelength analysis of SDSS J025214.67-002813.7, a system which has been previously classified as a binary AGN candidate based on periodic signals detected in the optical light curves. We use available radio-X-ray observations of the system to investigate the true accretion nature. Analyzing new observations from XMM-Newton and NuSTAR, we characterize the X-ray emission and search for evidence of circumbinary accretion. Although the 0.5-10 keV spectrum shows evidence of an additional soft emission component, possibly due to extended emission from hot nuclear gas, we find the spectral shape consistent with a single AGN. Compiling a full multi-wavelength SED, we also search for signs of circumbinary accretion, such as a &#34;notch&#34; in the continuum due to the presence of minidisks. We find that the radio-optical emission agrees with the SED of a standard, radio-quiet, AGN, however there is a large deficit in emission blueward of ~1400 A. Although this deficit in emission can plausibly be attributed to a binary AGN system, we find that the SED of SDSS J0252-0028 is better explained by emission from a reddened, single AGN. However, future studies on the expected hard X-ray emission associated with binary AGN (especially in the unequal-mass regime), will allow for more rigorous analyses of the binary AGN hypothesis.

preprint2021arXiv

Learning adaptive differential evolution algorithm from optimization experiences by policy gradient

Differential evolution is one of the most prestigious population-based stochastic optimization algorithm for black-box problems. The performance of a differential evolution algorithm depends highly on its mutation and crossover strategy and associated control parameters. However, the determination process for the most suitable parameter setting is troublesome and time-consuming. Adaptive control parameter methods that can adapt to problem landscape and optimization environment are more preferable than fixed parameter settings. This paper proposes a novel adaptive parameter control approach based on learning from the optimization experiences over a set of problems. In the approach, the parameter control is modeled as a finite-horizon Markov decision process. A reinforcement learning algorithm, named policy gradient, is applied to learn an agent (i.e. parameter controller) that can provide the control parameters of a proposed differential evolution adaptively during the search procedure. The differential evolution algorithm based on the learned agent is compared against nine well-known evolutionary algorithms on the CEC&#39;13 and CEC&#39;17 test suites. Experimental results show that the proposed algorithm performs competitively against these compared algorithms on the test suites.

preprint2021arXiv

MetaPhys: Few-Shot Adaptation for Non-Contact Physiological Measurement

There are large individual differences in physiological processes, making designing personalized health sensing algorithms challenging. Existing machine learning systems struggle to generalize well to unseen subjects or contexts and can often contain problematic biases. Video-based physiological measurement is not an exception. Therefore, learning personalized or customized models from a small number of unlabeled samples is very attractive as it would allow fast calibrations to improve generalization and help correct biases. In this paper, we present a novel meta-learning approach called MetaPhys for personalized video-based cardiac measurement for contactless pulse and heart rate monitoring. Our method uses only 18-seconds of video for customization and works effectively in both supervised and unsupervised manners. We evaluate our proposed approach on two benchmark datasets and demonstrate superior performance in cross-dataset evaluation with substantial reductions (42% to 44%) in errors compared with state-of-the-art approaches. We have also demonstrated our proposed method significantly helps reduce the bias in skin type.

preprint2021arXiv

Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement

Telehealth and remote health monitoring have become increasingly important during the SARS-CoV-2 pandemic and it is widely expected that this will have a lasting impact on healthcare practices. These tools can help reduce the risk of exposing patients and medical staff to infection, make healthcare services more accessible, and allow providers to see more patients. However, objective measurement of vital signs is challenging without direct contact with a patient. We present a video-based and on-device optical cardiopulmonary vital sign measurement approach. It leverages a novel multi-task temporal shift convolutional attention network (MTTS-CAN) and enables real-time cardiovascular and respiratory measurements on mobile platforms. We evaluate our system on an Advanced RISC Machine (ARM) CPU and achieve state-of-the-art accuracy while running at over 150 frames per second which enables real-time applications. Systematic experimentation on large benchmark datasets reveals that our approach leads to substantial (20%-50%) reductions in error and generalizes well across datasets.

preprint2021arXiv

OAAE: Adversarial Autoencoders for Novelty Detection in Multi-modal Normality Case via Orthogonalized Latent Space

Novelty detection using deep generative models such as autoencoder, generative adversarial networks mostly takes image reconstruction error as novelty score function. However, image data, high dimensional as it is, contains a lot of different features other than class information which makes models hard to detect novelty data. The problem gets harder in multi-modal normality case. To address this challenge, we propose a new way of measuring novelty score in multi-modal normality cases using orthogonalized latent space. Specifically, we employ orthogonal low-rank embedding in the latent space to disentangle the features in the latent space using mutual class information. With the orthogonalized latent space, novelty score is defined by the change of each latent vector. Proposed algorithm was compared to state-of-the-art novelty detection algorithms using GAN such as RaPP and OCGAN, and experimental results show that ours outperforms those algorithms.

preprint2021arXiv

On the Łojasiewicz Exponent of the Quadratic Sphere Constrained Optimization Problem

In this paper, we prove that the global version of the $Ł$ojasiewicz gradient inequality holds for quadratic sphere constrained optimization problem with exponent $θ=\frac{3}{4}$. An example from Ting Kei Pong shows that $θ=\frac{3}{4}$ is tight. This is the first $Ł$ojasiewicz gradient inequality established for the sphere constrained optimization problem with a linear term.

preprint2021arXiv

Online Mobile App Usage as an Indicator of Sleep Behavior and Job Performance

Sleep is critical to human function, mediating factors like memory, mood, energy, and alertness; therefore, it is commonly conjectured that a good night&#39;s sleep is important for job performance. However, both real-world sleep behavior and job performance are hard to measure at scale. In this work, we show that people&#39;s everyday interactions with online mobile apps can reveal insights into their job performance in real-world contexts. We present an observational study in which we objectively tracked the sleep behavior and job performance of salespeople (N = 15) and athletes (N = 19) for 18 months, using a mattress sensor and online mobile app. We first demonstrate that cumulative sleep measures are correlated with job performance metrics, showing that an hour of daily sleep loss for a week was associated with a 9.0% and 9.5% reduction in performance of salespeople and athletes, respectively. We then examine the utility of online app interaction time as a passively collectible and scalable performance indicator. We show that app interaction time is correlated with the performance of the athletes, but not the salespeople. To support that our app-based performance indicator captures meaningful variation in psychomotor function and is robust against potential confounds, we conducted a second study to evaluate the relationship between sleep behavior and app interaction time in a cohort of 274 participants. Using a generalized additive model to control for per-participant random effects, we demonstrate that participants who lost one hour of daily sleep for a week exhibited 5.0% slower app interaction times. We also find that app interaction time exhibits meaningful chronobiologically consistent correlations with sleep history, time awake, and circadian rhythms. Our findings reveal an opportunity for online app developers to generate new insights regarding cognition and productivity.

preprint2021arXiv

Spectrum Sharing for 6G Integrated Satellite-Terrestrial Communication Networks Based on NOMA and Cognitive Radio

The explosive growth of bandwidth hungry Internet applications has led to the rapid development of new generation mobile network technologies that are expected to provide broadband access to the Internet in a pervasive manner. For example, 6G networks are capable of providing high-speed network access by exploiting higher frequency spectrum; high-throughout satellite communication services are also adopted to achieve pervasive coverage in remote and isolated areas. In order to enable seamless access, Integrated Satellite-Terrestrial Communication Networks (ISTCN) has emerged as an important research area. ISTCN aims to provide high speed and pervasive network services by integrating broadband terrestrial mobile networks with satellite communication networks. As terrestrial mobile networks began to use higher frequency spectrum (between 3GHz to 40GHz) which overlaps with that of satellite communication (4GHz to 8GHz for C band and 26GHz to 40GHz for Ka band), there are opportunities and challenges. On one hand, satellite terminals can potentially access terrestrial networks in an integrated manner; on the other hand, there will be more congestion and interference in this spectrum, hence more efficient spectrum management techniques are required. In this paper, we propose a new technique to improve spectrum sharing performance by introducing Non-orthogonal Frequency Division Multiplexing (NOMA) and Cognitive Radio (CR) in the spectrum sharing of ISTCN. In essence, NOMA technology improves spectrum efficiency by allowing different users to transmit on the same carrier and distinguishing users by user power levels while CR technology improves spectrum efficiency through dynamic spectrum sharing. Furthermore, some open researches and challenges in ISTCN will be discussed.

preprint2021arXiv

SplitSR: An End-to-End Approach to Super-Resolution on Mobile Devices

Super-resolution (SR) is a coveted image processing technique for mobile apps ranging from the basic camera apps to mobile health. Existing SR algorithms rely on deep learning models with significant memory requirements, so they have yet to be deployed on mobile devices and instead operate in the cloud to achieve feasible inference time. This shortcoming prevents existing SR methods from being used in applications that require near real-time latency. In this work, we demonstrate state-of-the-art latency and accuracy for on-device super-resolution using a novel hybrid architecture called SplitSR and a novel lightweight residual block called SplitSRBlock. The SplitSRBlock supports channel-splitting, allowing the residual blocks to retain spatial information while reducing the computation in the channel dimension. SplitSR has a hybrid design consisting of standard convolutional blocks and lightweight residual blocks, allowing people to tune SplitSR for their computational budget. We evaluate our system on a low-end ARM CPU, demonstrating both higher accuracy and up to 5 times faster inference than previous approaches. We then deploy our model onto a smartphone in an app called ZoomSR to demonstrate the first-ever instance of on-device, deep learning-based SR. We conducted a user study with 15 participants to have them assess the perceived quality of images that were post-processed by SplitSR. Relative to bilinear interpolation -- the existing standard for on-device SR -- participants showed a statistically significant preference when looking at both images (Z=-9.270, p<0.01) and text (Z=-6.486, p<0.01).

preprint2021arXiv

Steady-State Analysis of Load Balancing with Coxian-$2$ Distributed Service Times

This paper studies load balancing for many-server ($N$ servers) systems. Each server has a buffer of size $b-1,$ and can have at most one job in service and $b-1$ jobs in the buffer. The service time of a job follows the Coxian-2 distribution. We focus on steady-state performance of load balancing policies in the heavy traffic regime such that the normalized load of system is $λ= 1 - N^{-α}$ for $0<α<0.5.$ We identify a set of policies that achieve asymptotic zero waiting. The set of policies include several classical policies such as join-the-shortest-queue (JSQ), join-the-idle-queue (JIQ), idle-one-first (I1F) and power-of-$d$-choices (Po$d$) with $d=O(N^α\log N)$. The proof of the main result is based on Stein&#39;s method and state space collapse. A key technical contribution of this paper is the iterative state space collapse approach that leads to a simple generator approximation when applying Stein&#39;s method.

preprint2021arXiv

Varstrometry for Off-nucleus and Dual sub-Kpc AGN (VODKA): Hubble Space Telescope Discovers Double Quasars

Dual supermassive black holes (SMBHs) at $\sim$kpc scales are the progenitor population of SMBH mergers and play an important role in understanding the pairing and dynamical evolution of massive black holes in galaxy mergers. Because of the stringent resolution requirement and the apparent rareness of these small-separation pairs, there are scarce observational constraints on this population, with few confirmed dual SMBHs at $<10$kpc separations at $z>1$. Here we present results from a pilot search for kpc-scale dual quasars selected with Gaia Data release 2 (DR2) astrometry and followed up with Hubble Space Telescope (HST) Wide Field Camera 3 dual-band (F475W and F814W) snapshot imaging. Our targets are quasars primarily selected with the varstrometry technique, i.e., light centroid jitter caused by asynchronous variability from both members in an unresolved quasar pair, supplemented by sub-arcsec pairs already resolved by Gaia DR2. We find an overall high fraction of HST-resolved pairs among the varstrometry-selected quasars (unresolved in Gaia DR2), $\sim 30-50\%$, increasing toward high redshift ($\sim 60-80\%$ at $z>1.5$). We discuss the nature of the 43 resolved sub-arcsec pairs based on HST and supplementary data. A substantial fraction ($\sim 40\%$) of these pairs are likely physical quasar pairs or gravitationally lensed quasars. We also discover a triple quasar candidate and a quadruply lensed quasar, which is among the smallest-separation quadruple lenses. These results provide important guidelines to improve varstrometry selection and follow-up confirmation of $\sim$kpc-scale dual SMBHs at high redshift.

preprint2021arXiv

Very Large Array imaging rules out precessing radio jets in three DES$-$SDSS-selected candidate periodic quasars

Periodic quasars have been suggested as candidates for hosting binary supermassive black holes (SMBHs), although alternative scenarios remain possible to explain the optical light curve periodicity. To test the alternative hypothesis of precessing radio jet, we present deep 6 GHz radio imaging conducted with NSF&#39;s Karl G. Jansky Very Large Array (VLA) in its C configuration for the three candidate periodic quasars, DES J024703.24$-$010032.0, DES J024944.66$-$000036.8, and DES J025214.67$-$002813.7. Our targets were selected based on their optical variability using 20-yr long multi-color light curves from the Dark Energy Survey (DES) and the Sloan Digital Sky Survey (SDSS). The new VLA observations show that all three periodic quasars are radio-quiet with the radio loudness parameters measured to be $R\equiv f_{6\,{\rm cm}}/f_{\rm 2500}$ of $\lesssim$1.0$-$1.5 and the $k$-corrected luminosities $νL_ν$[6 GHz] of $\lesssim$5$-$21 $\times$ 10$^{39}$ erg s$^{-1}$. They are in stark contrast to previously known periodic quasars proposed as binary SMBH candidates such as the blazar OJ 287 and PG1302$-$102. Our results rule out optical emission contributed from precessing radio jets as the origin of the optical periodicity in the three DES$-$SDSS-selected candidate periodic quasars. Future continued optical monitoring and complementary multi-wavelength observations are still needed to further test the binary SMBH hypothesis as well as other competing scenarios to explain the optical periodicity.

preprint2020arXiv

$B_{(s)} \to η_c(P,V)$ decays and effects of the next-to-leading order contributions in the perturbative QCD approach

By employing the perturbative QCD (PQCD) factorization approach, we studied the sixteen $B/B_s \to η_c (π, K, η^{(\prime)},ρ,K^*,ω,ϕ)$ decays with the inclusion of the currently known next-to-leading order (NLO) contributions. We found the following main points: (a) for the five measured $B \to η_c (K,K^*)$ and $B_s\to η_cϕ$ decays, the NLO contributions can provide $ (80-180)\%$ enhancements to the leading order (LO) PQCD predictions of their branching ratios, which play an important role to help us to interpret the data; (b) for the seven ratios $R_{1,\cdots,7}$ of the branching ratios defined among the properly selected pair of the considered decay modes, the PQCD predictions for the values of $R_{3,4,5}$ agree well with those currently available measurements from BaBar and Belle Collaboration; (c) for $B^0 \to η_c K_S^0$ decay, the PQCD predictions for both the direct and mixing induced CP asymmetries do agree very well with the measured values within errors; and (d) the PQCD predictions for ratios $R_{1,2}$ and $R_{6,7}$ also agree with the general expectations and will be tested by the future experiments.

preprint2020arXiv

2nd Place Scheme on Action Recognition Track of ECCV 2020 VIPriors Challenges: An Efficient Optical Flow Stream Guided Framework

To address the problem of training on small datasets for action recognition tasks, most prior works are either based on a large number of training samples or require pre-trained models transferred from other large datasets to tackle overfitting problems. However, it limits the research within organizations that have strong computational abilities. In this work, we try to propose a data-efficient framework that can train the model from scratch on small datasets while achieving promising results. Specifically, by introducing a 3D central difference convolution operation, we proposed a novel C3D neural network-based two-stream (Rank Pooling RGB and Optical Flow) framework for the task. The method is validated on the action recognition track of the ECCV 2020 VIPriors challenges and got the 2nd place (88.31%). It is proved that our method can achieve a promising result even without a pre-trained model on large scale datasets. The code will be released soon.

preprint2020arXiv

A Target-Agnostic Attack on Deep Models: Exploiting Security Vulnerabilities of Transfer Learning

Due to insufficient training data and the high computational cost to train a deep neural network from scratch, transfer learning has been extensively used in many deep-neural-network-based applications. A commonly used transfer learning approach involves taking a part of a pre-trained model, adding a few layers at the end, and re-training the new layers with a small dataset. This approach, while efficient and widely used, imposes a security vulnerability because the pre-trained model used in transfer learning is usually publicly available, including to potential attackers. In this paper, we show that without any additional knowledge other than the pre-trained model, an attacker can launch an effective and efficient brute force attack that can craft instances of input to trigger each target class with high confidence. We assume that the attacker has no access to any target-specific information, including samples from target classes, re-trained model, and probabilities assigned by Softmax to each class, and thus making the attack target-agnostic. These assumptions render all previous attack models inapplicable, to the best of our knowledge. To evaluate the proposed attack, we perform a set of experiments on face recognition and speech recognition tasks and show the effectiveness of the attack. Our work reveals a fundamental security weakness of the Softmax layer when used in transfer learning settings.

preprint2020arXiv

A Variational Approach to Unsupervised Sentiment Analysis

In this paper, we propose a variational approach to unsupervised sentiment analysis. Instead of using ground truth provided by domain experts, we use target-opinion word pairs as a supervision signal. For example, in a document snippet &#34;the room is big,&#34; (room, big) is a target-opinion word pair. These word pairs can be extracted by using dependency parsers and simple rules. Our objective function is to predict an opinion word given a target word while our ultimate goal is to learn a sentiment classifier. By introducing a latent variable, i.e., the sentiment polarity, to the objective function, we can inject the sentiment classifier to the objective function via the evidence lower bound. We can learn a sentiment classifier by optimizing the lower bound. We also impose sophisticated constraints on opinion words as regularization which encourages that if two documents have similar (dissimilar) opinion words, the sentiment classifiers should produce similar (different) probability distribution. We apply our method to sentiment analysis on customer reviews and clinical narratives. The experiment results show our method can outperform unsupervised baselines in sentiment analysis task on both domains, and our method obtains comparable results to the supervised method with hundreds of labels per aspect in customer reviews domain, and obtains comparable results to supervised methods in clinical narratives domain.

preprint2020arXiv

An Opportunistic Bandit Approach for User Interface Experimentation

Facing growing competition from online rivals, the retail industry is increasingly investing in their online shopping platforms to win the high-stake battle of customer&#39; loyalty. User experience is playing an essential role in this competition, and retailers are continuously experimenting and optimizing their user interface for better user experience. The cost of experimentation is dominated by the opportunity cost of providing a suboptimal service to the customers. Through this paper, we demonstrate the effectiveness of opportunistic bandits to make the experiments as inexpensive as possible using real online retail data. In fact, we model user interface experimentation as an opportunistic bandit problem, in which the cost of exploration varies under a factor extracted from customer features. We achieve significant regret reduction by mitigating costly exploration and providing extra contextual information that helps to guide the testing process. Moreover, we analyze the advantages and challenges of using opportunistic bandits for online retail experimentation.

preprint2020arXiv

An orthogonalization-free parallelizable framework for all-electron calculations in density functional theory

All-electron calculations play an important role in density functional theory, in which improving computational efficiency is one of the most needed and challenging tasks. In the model formulations, both nonlinear eigenvalue problem and total energy minimization problem pursue orthogonal solutions. Most existing algorithms for solving these two models invoke orthogonalization process either explicitly or implicitly in each iteration. Their efficiency suffers from this process in view of its cubic complexity and low parallel scalability in terms of the number of electrons for large scale systems. To break through this bottleneck, we propose an orthogonalization-free algorithm framework based on the total energy minimization problem. It is shown that the desired orthogonality can be gradually achieved without invoking orthogonalization in each iteration. Moreover, this framework fully consists of Basic Linear Algebra Subprograms (BLAS) operations and thus can be naturally parallelized. The global convergence of the proposed algorithm is established. We also present a precondition technique which can dramatically accelerate the convergence of the algorithm. The numerical experiments on all-electron calculations show the efficiency and high scalability of the proposed algorithm.

preprint2020arXiv

ASER: A Large-scale Eventuality Knowledge Graph

Understanding human&#39;s language requires complex world knowledge. However, existing large-scale knowledge graphs mainly focus on knowledge about entities while ignoring knowledge about activities, states, or events, which are used to describe how entities or things act in the real world. To fill this gap, we develop ASER (activities, states, events, and their relations), a large-scale eventuality knowledge graph extracted from more than 11-billion-token unstructured textual data. ASER contains 15 relation types belonging to five categories, 194-million unique eventualities, and 64-million unique edges among them. Both intrinsic and extrinsic evaluations demonstrate the quality and effectiveness of ASER.

preprint2020arXiv

Branching Fractions and CP Asymmetries of the Quasi-Two-Body Decays in $B_{s} \to K^0(\overline K^0)K^\pm π^\mp$ within PQCD Approach

Motivated by the first untagged decay-time-integrated amplitude analysis of $B_s \to K_SK^{\mp}π^{\pm}$ decays performed by LHCb collaboration, where the decay amplitudes are modeled to contain the resonant contributions from intermediate resonances $K^*(892)$, $K_0^*(1430)$ and $K_2^*(1430)$, we comprehensively investigate the quasi-two-body $B_{s} \to K^0/\overline{\kern -0.2em K}^0 K^{\pm}π^{\mp}$ decays, and calculate the branching fractions and the time-dependent $CP$ asymmetries within the perturbative QCD approach based on the $k_T$ factorization. In the quasi-two-body space region the calculated branching fractions with the considered intermediate resonances are in good agreement with the experimental results of LHCb by adopting proper $Kπ$ pair wave function, describing the interaction between the kaon and pion in the $Kπ$ pair. Furthermore,within the obtained branching fractions of the quasi-two-body decays, we also calculate the branching fractions of corresponding two-body decays, and the results consist with the LHCb measurements and the earlier studies with errors. For these considered decays, since the final states are not flavour-specific, the time-dependent $CP$ could be measured. We calculate six $CP$-violation observables, which can be tested in the ongoing LHCb experiment.

preprint2020arXiv

Dark Energy Survey Identification of A Low-Mass Active Galactic Nucleus at Redshift 0.823 from Optical Variability

We report the identification of a low-mass AGN, DES J0218$-$0430, in a redshift $z = 0.823$ galaxy in the Dark Energy Survey (DES) Supernova field. We select DES J0218$-$0430 as an AGN candidate by characterizing its long-term optical variability alone based on DES optical broad-band light curves spanning over 6 years. An archival optical spectrum from the fourth phase of the Sloan Digital Sky Survey shows both broad Mg II and broad H$β$ lines, confirming its nature as a broad-line AGN. Archival XMM-Newton X-ray observations suggest an intrinsic hard X-ray luminosity of $L_{\rm 2-12\,keV}\sim7.6\pm0.4\times10^{43}$ erg s$^{-1}$, which exceeds those of the most X-ray luminous starburst galaxies, in support of an AGN driving the optical variability. Based on the broad H$β$ from SDSS spectrum, we estimate a virial BH mass of $M_{\bullet}\approx10^{6.43}$-$10^{6.72}M_{\odot}$ (with the error denoting 1$σ$ statistical uncertainties only), consistent with the estimation from OzDES, making it the lowest mass AGN with redshift $>$ 0.4 detected in optical. We estimate the host galaxy stellar mass to be $M_{\ast}\sim10^{10.5\pm0.3}M_{\odot}$ based on modeling the multi-wavelength spectral energy distribution. DES J0218$-$0430 extends the $M_{\bullet}$-$M_{\ast}$ relation observed in luminous AGNs at $z\sim1$ to masses lower than being probed by previous work. Our work demonstrates the feasibility of using optical variability to identify low-mass AGNs at higher redshift in deeper synoptic surveys with direct implications for the upcoming Legacy Survey of Space and Time at Vera C. Rubin Observatory.

preprint2020arXiv

Decomposing Word Embedding with the Capsule Network

Word sense disambiguation tries to learn the appropriate sense of an ambiguous word in a given context. The existing pre-trained language methods and the methods based on multi-embeddings of word did not explore the power of the unsupervised word embedding sufficiently. In this paper, we discuss a capsule network-based approach, taking advantage of capsule&#39;s potential for recognizing highly overlapping features and dealing with segmentation. We propose a Capsule network-based method to Decompose the unsupervised word Embedding of an ambiguous word into context specific Sense embedding, called CapsDecE2S. In this approach, the unsupervised ambiguous embedding is fed into capsule network to produce its multiple morpheme-like vectors, which are defined as the basic semantic language units of meaning. With attention operations, CapsDecE2S integrates the word context to reconstruct the multiple morpheme-like vectors into the context-specific sense embedding. To train CapsDecE2S, we propose a sense matching training method. In this method, we convert the sense learning into a binary classification that explicitly learns the relation between senses by the label of matching and non-matching. The CapsDecE2S was experimentally evaluated on two sense learning tasks, i.e., word in context and word sense disambiguation. Results on two public corpora Word-in-Context and English all-words Word Sense Disambiguation show that, the CapsDecE2S model achieves the new state-of-the-art for the word in context and word sense disambiguation tasks.

preprint2020arXiv

Deep Low-rank Prior in Dynamic MR Imaging

The deep learning methods have achieved attractive performance in dynamic MR cine imaging. However, all of these methods are only driven by the sparse prior of MR images, while the important low-rank (LR) prior of dynamic MR cine images is not explored, which limits the further improvements on dynamic MR reconstruction. In this paper, a learned singular value thresholding (Learned-SVT) operation is proposed to explore deep low-rank prior in dynamic MR imaging for obtaining improved reconstruction results. In particular, we come up with two novel and distinct schemes to introduce the learnable low-rank prior into deep network architectures in an unrolling manner and a plug-and-play manner respectively. In the unrolling manner, we put forward a model-based unrolling sparse and low-rank network for dynamic MR imaging, dubbed SLR-Net. The SLR-Net is defined over a deep network flow graph, which is unrolled from the iterative procedures in the Iterative Shrinkage-Thresholding Algorithm (ISTA) for optimizing a sparse and low-rank based dynamic MRI model. In the plug-and-play manner, we present a plug-and-play LR network module that can be easily embedded into any other dynamic MR neural networks without changing the network paradigm. Experimental results show that both schemes can further improve the state-of-the-art CS methods, such as k-t SLR, and sparsity-driven deep learning-based methods, such as DC-CNN and CRNN, both qualitatively and quantitatively.

preprint2020arXiv

Design admissibility and de la Garza phenomenon in multi-factor experiments

The determination of an optimal design for a given regression problem is an intricate optimization problem, especially for models with multivariate predictors. Design admissibility and invariance are main tools to reduce the complexity of the optimization problem and have been successfully applied for models with univariate predictors. In particular several authors have developed sufficient conditions for the existence of saturated designs in univariate models, where the number of support points of the optimal design equals the number of parameters. These results generalize the celebrated de la Garza phenomenon (de la Garza, 1954) which states that for a polynomial regression model of degree $p-1$ any optimal design can be based on at most $p$ points. This paper provides - for the first time - extensions of these results for models with a multivariate predictor. In particular we study a geometric characterization of the support points of an optimal design to provide sufficient conditions for the occurrence of the de la Garza phenomenon in models with multivariate predictors and characterize properties of admissible designs in terms of admissibility of designs in conditional univariate regression models.

preprint2020arXiv

Dust Reverberation Mapping in Distant Quasars from Optical and Mid-Infrared Imaging Surveys

The size of the dust torus in Active Galactic Nuclei (AGN) and their high-luminosity counterparts, quasars, can be inferred from the time delay between UV/optical accretion disk continuum variability and the response in the mid-infrared (MIR) torus emission. This dust reverberation mapping (RM) technique has been successfully applied to $\sim 70$ $z\lesssim 0.3$ AGN and quasars. Here we present first results of our dust RM program for distant quasars covered in the SDSS Stripe 82 region combining $\sim 20$-yr ground-based optical light curves with 10-yr MIR light curves from the WISE satellite. We measure a high-fidelity lag between W1-band (3.4 $μ$m) and $g$ band for 587 quasars over $0.3\lesssim z\lesssim 2$ ($\left<z\right>\sim 0.8$) and two orders of magnitude in quasar luminosity. They tightly follow (intrinsic scatter $\sim 0.17$ dex in lag) the IR lag-luminosity relation observed for $z<0.3$ AGN, revealing a remarkable size-luminosity relation for the dust torus over more than four decades in AGN luminosity, with little dependence on additional quasar properties such as Eddington ratio and variability amplitude. This study motivates further investigations in the utility of dust RM for cosmology, and strongly endorses a compelling science case for the combined 10-yr Vera C. Rubin Observatory Legacy Survey of Space and Time (optical) and 5-yr Nancy Grace Roman Space Telescope 2$μ$m light curves in a deep survey for low-redshift AGN dust RM with much lower luminosities and shorter, measurable IR lags. The compiled optical and MIR light curves for 7,384 quasars in our parent sample are made public with this work.

preprint2020arXiv

Electric-field-controllable high-spin SrRuO3 driven by a solid ionic junction

Controlling magnetism and spin structures in strongly correlated systems by using electric field is of fundamental importance but challenging. Here, a high-spin ruthenate phase is achieved via a solid ionic chemical junction at SrRuO3/SrTiO3 interface with distinct formation energies and diffusion barriers of oxygen vacancies, analogue to electronic band alignment in semiconductor heterojunction. Oxygen vacancies trapped within this interfacial SrRuO3 reconstruct Ru-4d electronic structure and orbital occupancy, leading to an enhanced magnetic moment. Furthermore, an interfacial magnetic phase can be switched reversibly by electric-field-rectifying oxygen migration in a solid-state ionic gating device, providing a framework for atomic design of functionalities in strongly correlated oxides using a way of solid chemistry.

preprint2020arXiv

ENAS4D: Efficient Multi-stage CNN Architecture Search for Dynamic Inference

Dynamic inference is a feasible way to reduce the computational cost of convolutional neural network(CNN), which can dynamically adjust the computation for each input sample. One of the ways to achieve dynamic inference is to use multi-stage neural network, which contains a sub-network with prediction layer at each stage. The inference of a input sample can exit from early stage if the prediction of the stage is confident enough. However, design a multi-stage CNN architecture is a non-trivial task. In this paper, we introduce a general framework, ENAS4D, which can efficiently search for optimal multi-stage CNN architecture for dynamic inference in a well-designed search space. Firstly, we propose a method to construct the search space with multi-stage convolution. The search space include different numbers of layers, different kernel sizes and different numbers of channels for each stage and the resolution of input samples. Then, we train a once-for-all network that supports to sample diverse multi-stage CNN architecture. A specialized multi-stage network can be obtained from the once-for-all network without additional training. Finally, we devise a method to efficiently search for the optimal multi-stage network that trades the accuracy off the computational cost taking the advantage of once-for-all network. The experiments on the ImageNet classification task demonstrate that the multi-stage CNNs searched by ENAS4D consistently outperform the state-of-the-art method for dyanmic inference. In particular, the network achieves 74.4% ImageNet top-1 accuracy under 185M average MACs.

preprint2020arXiv

Extreme-Scale Density Functional Theory High Performance Computing of DGDFT for Tens of Thousands of Atoms using Millions of Cores on Sunway TaihuLight

High performance computing (HPC) is a powerful tool to accelerate the Kohn-Sham density functional theory (KS-DFT) calculations on modern heterogeneous supercomputers. Here, we describe a massively extreme-scale parallel and portable implementation of discontinuous Galerkin density functional theory (DGDFT) method on the Sunway TaihuLight supercomputer. The DGDFT method uses the adaptive local basis (ALB) functions generated on-the-fly during the self-consistent field (SCF) iteration to solve the KS equations with the high precision comparable to that of plane-wave basis set. In particular, the DGDFT method adopts a two-level parallelization strategy that makes use of different types of data distribution, task scheduling, and data communication schemes, and combines with the feature of master-slave multi-thread heterogeneous parallelism of SW26010 processor, resulting in extreme-scale HPC KS-DFT calculations on the Sunway TaihuLight supercomputer. We show that the DGDFT method can scale up to 8,519,680 processing cores (131,072 core groups) on the Sunway TaihuLight supercomputer for investigating the electronic structures of two-dimensional (2D) metallic graphene systems containing tens of thousands of carbon atoms.

preprint2020arXiv

Getting Ready for LISA: The Data, Support and Preparation Needed to Maximize US Participation in Space-Based Gravitational Wave Science

The NASA LISA Study Team was tasked to study how NASA might support US scientists to participate and maximize the science return from the Laser Interferometer Space Antenna (LISA) mission. LISA is gravitational wave observatory led by ESA with NASA as a junior partner, and is scheduled to launch in 2034. Among our findings: LISA science productivity is greatly enhanced by a full-featured US science center and an open access data model. As other major missions have demonstrated, a science center acts as both a locus and an amplifier of research innovation, data analysis, user support, user training and user interaction. In its most basic function, a US Science Center could facilitate entry into LISA science by hosting a Data Processing Center and a portal for the US community to access LISA data products. However, an enhanced LISA Science Center could: support one of the parallel independent processing pipelines required for data product validation; stimulate the high level of research on data analysis that LISA demands; support users unfamiliar with a novel observatory; facilitate astrophysics and fundamental research; provide an interface into the subtleties of the instrument to validate extraordinary discoveries; train new users; and expand the research community through guest investigator, postdoc and student programs. Establishing a US LISA Science Center well before launch can have a beneficial impact on the participation of the broader astronomical community by providing training, hosting topical workshops, disseminating mock catalogs, software pipelines, and documentation. Past experience indicates that successful science centers are established several years before launch; this early adoption model may be especially relevant for a pioneering mission like LISA.

preprint2020arXiv

Giant enhancement in the thermal responsivity of microelectromechanical resonators by internal mode coupling

We report on a giant enhancement in the thermal responsivity of the doubly-clamped GaAs microelectromechanical (MEMS) beam resonators by using the internal mode coupling effect. This is achieved by coupling the fundamental bending mode with the fundamental torsional mode of the MEMS beam resonators through the cubic Duffing nonlinearity. In the mode coupling regime, we have found that, when the input heat to the MEMS resonators is modulated at a particular frequency, the resonance frequency shift caused by heating can be enhanced by almost two orders of magnitude. The observed effect is promising for realizing high-sensitivity thermal sensing by using MEMS resonators, such as ultrasensitive terahertz detection at room temperature.

preprint2020arXiv

Global Solutions to Compressible Navier-Stokes Equations With Spherically Symmetric Motion and Free Boundary

This work is devoted to study the global existence of strong and classical solutions to compressible Navier-Stokes equations with or without density jump on the moving boundary for spherically symmetric motion. We establish a unified method to track the propagation of regularity of strong and classical solutions which works for the cases when density connects to vacuum continuously and with a jump simultaneously. The result we obtain is able to deal with both strong solutions with physical vacuum for which the sound speed is $1/2$-Hölder continuous across the boundary, and classical solutions with physical vacuum when $ 1 < γ< 3 $. In contrast to the previous results of global weak solutions, we track the regularity globally-in-time up to the symmetric center and the moving boundary. In particular, the free boundary can be traced.

preprint2020arXiv

High-$T_c$ Superconductor Fe(Se,Te) Monolayer: an Intrinsic, Scalable and Electrically-tunable Majorana Platform

A monolayer of the high-$T_c$ superconductor FeTe$_{1-x}$Se$_x$ has been predicted to realize a topologically non-trivial state with helical edge modes at its boundary, providing a novel intrinsic system to search for topological superconductivity and Majorana zero modes. Evidence in favor of a topological phase transition and helical edge modes has been identified in recent experiments \cite{Peng2019}. We propose to create Majorana zero modes by applying an in-plane magnetic field to the FeTe$_{1-x}$Se$_x$ monolayer and by tuning the local chemical potential via electric gating. Owing to the anisotropic magnetic couplings on edges from a topological band inversion, an in-plane magnetic field drives the system into an intrinsic high-order topological superconductor phase with Majorana corner modes, without fabricating heterostructures. Furthermore, we demonstrate that Majorana zero modes can occur at other different locations, including the domain wall of chemical potentials at one edge and certain type of tri-junction in the 2D bulk. Our study not only demonstrates FeTe$_{1-x}$Se$_x$ monolayer as a promising Majorana platform with scalability and electrical tunability and within reach of contemporary experimental capability, but also provides a general principle to search for realistic realization of high-order topological superconductivity.

preprint2020arXiv

How to Achieve High Classification Accuracy with Just a Few Labels: A Semi-supervised Approach Using Sampled Packets

Network traffic classification, which has numerous applications from security to billing and network provisioning, has become a cornerstone of today&#39;s computer networks. Previous studies have developed traffic classification techniques using classical machine learning algorithms and deep learning methods when large quantities of labeled data are available. However, capturing large labeled datasets is a cumbersome and time-consuming process. In this paper, we propose a semi-supervised approach that obviates the need for large labeled datasets. We first pre-train a model on a large unlabeled dataset where the input is the time series features of a few sampled packets. Then the learned weights are transferred to a new model that is re-trained on a small labeled dataset. We show that our semi-supervised approach achieves almost the same accuracy as a fully-supervised method with a large labeled dataset, though we use only 20 samples per class. In tests based on a dataset generated from the more challenging QUIC protocol, our approach yields 98% accuracy. To show its efficacy, we also test our approach on two public datasets. Moreover, we study three different sampling techniques and demonstrate that sampling packets from an arbitrary portion of a flow is sufficient for classification.

preprint2020arXiv

Knot theory for two-band model of two-dimensional square lattice with high topological numbers

A knot theory for two-dimensional square lattice is proposed, which sheds light on design of new two-dimensional material with high topological numbers. We consider a two-band model, focusing on the Hall conductance σxy = e^2/hbar*P, where P is a topological number, the so-called Pontrjagin index. By re-interpreting the periodic momentum components kx and ky as the string parameters of two entangled knots, we discover that P becomes the Gauss linking number between the knots. This leads to a successful re-derivation of the typical P-evaluations in literature: P = 0;{\pm}1. Furthermore, with the aid of this explicit knot theoretical picture we modify the two-band model to achieve higher topological numbers, P = 0;{\pm}1;{\pm}2.

preprint2020arXiv

Learning Diverse Fashion Collocation by Neural Graph Filtering

Fashion recommendation systems are highly desired by customers to find visually-collocated fashion items, such as clothes, shoes, bags, etc. While existing methods demonstrate promising results, they remain lacking in flexibility and diversity, e.g. assuming a fixed number of items or favoring safe but boring recommendations. In this paper, we propose a novel fashion collocation framework, Neural Graph Filtering, that models a flexible set of fashion items via a graph neural network. Specifically, we consider the visual embeddings of each garment as a node in the graph, and describe the inter-garment relationship as the edge between nodes. By applying symmetric operations on the edge vectors, this framework allows varying numbers of inputs/outputs and is invariant to their ordering. We further include a style classifier augmented with focal loss to enable the collocation of significantly diverse styles, which are inherently imbalanced in the training set. To facilitate a comprehensive study on diverse fashion collocation, we reorganize Amazon Fashion dataset with carefully designed evaluation protocols. We evaluate the proposed approach on three popular benchmarks, the Polyvore dataset, the Polyvore-D dataset, and our reorganized Amazon Fashion dataset. Extensive experimental results show that our approach significantly outperforms the state-of-the-art methods with over 10% improvements on the standard AUC metric on the established tasks. More importantly, 82.5% of the users prefer our diverse-style recommendations over other alternatives in a real-world perception study.

preprint2020arXiv

MMFashion: An Open-Source Toolbox for Visual Fashion Analysis

We present MMFashion, a comprehensive, flexible and user-friendly open-source visual fashion analysis toolbox based on PyTorch. This toolbox supports a wide spectrum of fashion analysis tasks, including Fashion Attribute Prediction, Fashion Recognition and Retrieval, Fashion Landmark Detection, Fashion Parsing and Segmentation and Fashion Compatibility and Recommendation. It covers almost all the mainstream tasks in fashion analysis community. MMFashion has several appealing properties. Firstly, MMFashion follows the principle of modular design. The framework is decomposed into different components so that it is easily extensible for diverse customized modules. In addition, detailed documentations, demo scripts and off-the-shelf models are available, which ease the burden of layman users to leverage the recent advances in deep learning-based fashion analysis. Our proposed MMFashion is currently the most complete platform for visual fashion analysis in deep learning era, with more functionalities to be added. This toolbox and the benchmark could serve the flourishing research community by providing a flexible toolkit to deploy existing models and develop new ideas and approaches. We welcome all contributions to this still-growing efforts towards open science: https://github.com/open-mmlab/mmfashion.

preprint2020arXiv

Multitask Learning for Network Traffic Classification

Traffic classification has various applications in today&#39;s Internet, from resource allocation, billing and QoS purposes in ISPs to firewall and malware detection in clients. Classical machine learning algorithms and deep learning models have been widely used to solve the traffic classification task. However, training such models requires a large amount of labeled data. Labeling data is often the most difficult and time-consuming process in building a classifier. To solve this challenge, we reformulate the traffic classification into a multi-task learning framework where bandwidth requirement and duration of a flow are predicted along with the traffic class. The motivation of this approach is twofold: First, bandwidth requirement and duration are useful in many applications, including routing, resource allocation, and QoS provisioning. Second, these two values can be obtained from each flow easily without the need for human labeling or capturing flows in a controlled and isolated environment. We show that with a large amount of easily obtainable data samples for bandwidth and duration prediction tasks, and only a few data samples for the traffic classification task, one can achieve high accuracy. We conduct two experiment with ISCX and QUIC public datasets and show the efficacy of our approach.

preprint2020arXiv

Neural Subgraph Isomorphism Counting

In this paper, we study a new graph learning problem: learning to count subgraph isomorphisms. Different from other traditional graph learning problems such as node classification and link prediction, subgraph isomorphism counting is NP-complete and requires more global inference to oversee the whole graph. To make it scalable for large-scale graphs and patterns, we propose a learning framework which augments different representation learning architectures and iteratively attends pattern and target data graphs to memorize subgraph isomorphisms for the global counting. We develop both small graphs (<= 1,024 subgraph isomorphisms in each) and large graphs (<= 4,096 subgraph isomorphisms in each) sets to evaluate different models. A mutagenic compound dataset, MUTAG, is also used to evaluate neural models and demonstrate the success of transfer learning. While the learning based approach is inexact, we are able to generalize to count large patterns and data graphs in linear time compared to the exponential time of the original NP-complete problem. Experimental results show that learning based subgraph isomorphism counting can speed up the traditional algorithm, VF2, 10-1,000 times with acceptable errors. Domain adaptation based on fine-tuning also shows the usefulness of our approach in real-world applications.

preprint2020arXiv

On the Importance of Word and Sentence Representation Learning in Implicit Discourse Relation Classification

Implicit discourse relation classification is one of the most difficult parts in shallow discourse parsing as the relation prediction without explicit connectives requires the language understanding at both the text span level and the sentence level. Previous studies mainly focus on the interactions between two arguments. We argue that a powerful contextualized representation module, a bilateral multi-perspective matching module, and a global information fusion module are all important to implicit discourse analysis. We propose a novel model to combine these modules together. Extensive experiments show that our proposed model outperforms BERT and other state-of-the-art systems on the PDTB dataset by around 8% and CoNLL 2016 datasets around 16%. We also analyze the effectiveness of different modules in the implicit discourse relation classification task and demonstrate how different levels of representation learning can affect the results.

preprint2020arXiv

Optical Variability of the Dwarf AGN NGC 4395 from the Transiting Exoplanet Survey Satellite

We present optical light curves from the Transiting Exoplanet Survey Satellite (TESS) for the archetypical dwarf active galactic nucleus (AGN) in the nearby galaxy NGC 4395 hosting a $\sim 10^5\,M_\odot$ supermassive black hole (SMBH). Significant variability is detected on timescales from weeks to hours before reaching the background noise level. The $\sim$month-long, 30 minute-cadence, high-precision TESS light curve can be well fit by a simple damped random walk (DRW) model, with the damping timescale $τ_{\rm DRW}$ constrained to be $2.3_{-0.7}^{+1.8}$~days ($1σ$). NGC 4395 lies almost exactly on the extrapolation of the $τ_{\rm DRW}-M_{\rm BH}$ relation measured for AGNs with BH masses that are more than three orders of magnitude larger. The optical variability periodogram can be well fit by a broken power law with the high-frequency slope ($-1.88\pm0.15$) and the characteristic timescale ($τ_{\rm br}\equiv 1/(2πf_{\rm br})=1.4_{-0.5}^{+1.9}\,$days) consistent with the DRW model within 1$σ$. This work demonstrates the power of TESS light curves in identifying low-mass accreting SMBHs with optical variability, and a potential global $τ_{\rm DRW}-M_{\rm BH}$ relation that can be used to estimate SMBH masses with optical variability measurements.

preprint2020arXiv

Poisson stable solutions for stochastic differential equations with Lévy noise

In this paper, we use a unified framework to study Poisson stable (including stationary, periodic, quasi-periodic, almost periodic, almost automorphic, Birkhoff recurrent, almost recurrent in the sense of Bebutov, Levitan almost periodic, pseudo-periodic, pseudo-recurrent and Poisson stable) solutions for semilinear stochastic differential equations driven by infinite dimensional Lévy noise with large jumps. Under suitable conditions on drift, diffusion and jump coefficients, we prove that there exist solutions which inherit the Poisson stability of coefficients. Further we show that these solutions are globally asymptotically stable in square-mean sense. Finally, we illustrate our theoretical results by several examples.

preprint2020arXiv

Positive Contrast Susceptibility MR Imaging Using GPU-based Primal-Dual Algorithm

The susceptibility-based positive contrast MR technique was applied to estimate arbitrary magnetic susceptibility distributions of the metallic devices using a kernel deconvolution algorithm with a regularized L-1 minimization.Previously, the first-order primal-dual (PD) algorithm could provide a faster reconstruction time to solve the L-1 minimization, compared with other methods. Here, we propose to accelerate the PD algorithm of the positive contrast image using the multi-core multi-thread feature of graphics processor units (GPUs). The some experimental results showed that the GPU-based PD algorithm could achieve comparable accuracy of the metallic interventional devices in positive contrast imaging with less computational time. And the GPU-based PD approach was 4~15 times faster than the previous CPU-based scheme.

preprint2020arXiv

Real-Space Imaging of the Ordered Small Molecule Orientations in Porous Frameworks by Electron Microscopy

The real-space imaging of small molecules is always challenging under the electron microscopes, but highly demanded for investigating various nanoscale interactions, such as hydrogen bond and van der Waals (vdW) force. Especially, identifying the host-guest interactions in porous materials directly at the molecular level will bring a deeper insight into the behaviors of guest molecules during the sorption, catalysis, gas separation and energy storage. In this work, we directly resolved the ordered configurations of p-xylenes (PXs) adsorbed in ZSM-5 frameworks by the scanning transmission electron microscopy (STEM) with the integrated differential phase contrast (iDPC) technique to identify the host-guest vdW interactions. Based on these observations, we revealed that the PXs in one straight channel modified the channel geometry with a coherent orientation. And the adjacent straight channels were deformed up to 8.8% along the different directions corresponding to three dominant PX configurations, resulting a negligible overall expansion of ZSM-5 lattices. Then, we could also image the disorder and desorption of PXs in ZSM-5 channels during the in situ heating. This work not only helped us to study the host-guest vdW interactions and the sorption behaviors of PXs in ZSM-5, but also provided an efficient tool for further imaging and studying other single-molecule behaviors under STEMs.

preprint2020arXiv

Resonant Contributions to Three-body $B\to KKK$ Decays in Perturbative QCD Approach

In this work, we study the ($S$, $P$ and $D$)-wave $K^+K^-$ contributions to $B\to KKK$ decays in the perturbative QCD approach at leading order. Within the two-meson wave functions describing the nonperturbative dynamics in the kaon-pair for different waves, we calculate the branching fractions and the direct $CP$ asymmetries of these decay modes in the corresponding resonance regions. Most of our numerical results are well consistent with the current measurements. We note that the narrow-width approximation is invalid in the quasi-two-body decays $B\to Kf_0(980)\to KKK$. For other decays, under the narrow-width approximation we can extract the branching fractions of the corresponding two-body decays involving the intermediate resonant states, and the related branching fractions agree with the current experimental data well. Furthermore, we also predict the corresponding quasi-two-body decays $B\to Kπ^+π^-$, which are expected to be measured in the ongoing LHCb and Belle-II experiments.

preprint2020arXiv

Study of Quasi-two-body $B_{(s)}\to ϕ(f_0(980)/f_2(1270)\to)ππ$ Decays in Perturbative QCD Approach

In 2017, LHCb collaboration reported their first observation of the rare decays $B_s \to ϕ(f_0(980)$ $/f_2(1270) \to ) π^+π^-$ and the evidence of $B^0 \to ϕ(f_0(980)/f_2(1270)\to)π^+π^-$. Motivated by this, we study these quasi-two-body decays in the perturbative QCD approach. The branching fractions, $CP$ asymmetries and the polarization fractions are calculated. We find that within the appropriate two-meson wave functions, the calculated branching fractions are in agreement with the measurements of LHCb. Based on the narrow-width approximation, We also calculate the branching fractions of the quasi-two-body $B_{d,s}\to ϕ(f_0(980)/f_2(1270)\to) π^0π^0$ and $B_{d,s}\to ϕ(f_2(1270)\to) K^+K^-$, and hope the predictions to be tested in the ongoing LHCb and Belle II experiments. Moreover, the processes $B_{d,s}\to ϕf_2(1270)$ are also analyzed under this approximation. We note that the $CP$ asymmetries of these decays are very small, because these decays are either penguin dominant or pure penguin processes.

preprint2020arXiv

Superconducting proximity effect in a transparent van der Waals superconductor-metal junction

We report on Andreev reflections at clean NbSe2-bilayer graphene junctions. The high transparency of the junction, which manifests as a large conductance enhancement of up to 1.8, enables us to see clear evidence of a proximity-induced superconducting gap in bilayer graphene and two Andreev reflections through a vertical NbSe2-graphene and a lateral graphene-graphene junction respectively. Quantum transport simulations capture the complexity of the experimental data and illuminate the impact of various microscopic parameters on the transmission of the junction. Our work establishes the practice and understanding of an all-van-der-Waals, high-performance superconducting junction. The realization of a highly transparent proximized graphene-graphene junction opens up possibilities to engineer emergent quantum phenomena.

preprint2020arXiv

Symplectic Runge-Kutta discretization of a regularized forward-backward sweep iteration for optimal control problems

Li, Chen, Tai & E. (J. Machine Learning Research, 2018) have proposed a regularization of the forward-backward sweep iteration for solving the Pontryagin maximum principle in optimal control problems. The authors prove the global convergence of the iteration in the continuous time case. In this article we show that their proof can be extended to the case of numerical discretization by symplectic Runge-Kutta pairs. We demonstrate the convergence with a simple numerical experiment.

preprint2020arXiv

The Curious Case of PHL 293B: A Long-Lived Transient in a Metal-Poor Blue Compact Dwarf Galaxy

We report on small-amplitude optical variability and recent dissipation of the unusually persistent broad emission lines in the blue compact dwarf galaxy PHL 293B. The galaxy&#39;s unusual spectral features (P Cygni-like profiles with $\sim$800 km s$^{-1}$ blueshifted absorption lines) have resulted in conflicting interpretations of the nature of this source in the literature. However, analysis of new Gemini spectroscopy reveals the broad emission has begun to fade after being persistent for over a decade prior. Precise difference imaging light curves constructed with the Sloan Digital Sky Survey and the Dark Energy Survey reveal small-amplitude optical variability of $\sim$0.1 mag in the g band offset by $100\pm21$ pc from the brightest pixel of the host. The light curve is well-described by an active galactic nuclei (AGN)-like damped random walk process. However, we conclude that the origin of the optical variability and spectral features of PHL 293B is due to a long-lived stellar transient, likely a Type IIn supernova or non-terminal outburst, mimicking long-term AGN-like variability. This work highlights the challenges of discriminating between scenarios in such extreme environments, relevant to searches for AGNs in dwarf galaxies. This is the second long-lived transient discovered in a blue compact dwarf, after SDSS1133. Our result implies such long-lived stellar transients may be more common in metal-deficient galaxies. Systematic searches for low-level variability in dwarf galaxies will be possible with the upcoming Legacy Survey of Space and Time at Vera C. Rubin Observatory.

preprint2020arXiv

XGPT: Cross-modal Generative Pre-Training for Image Captioning

While many BERT-based cross-modal pre-trained models produce excellent results on downstream understanding tasks like image-text retrieval and VQA, they cannot be applied to generation tasks directly. In this paper, we propose XGPT, a new method of Cross-modal Generative Pre-Training for Image Captioning that is designed to pre-train text-to-image caption generators through three novel generation tasks, including Image-conditioned Masked Language Modeling (IMLM), Image-conditioned Denoising Autoencoding (IDA), and Text-conditioned Image Feature Generation (TIFG). As a result, the pre-trained XGPT can be fine-tuned without any task-specific architecture modifications to create state-of-the-art models for image captioning. Experiments show that XGPT obtains new state-of-the-art results on the benchmark datasets, including COCO Captions and Flickr30k Captions. We also use XGPT to generate new image captions as data augmentation for the image retrieval task and achieve significant improvement on all recall metrics.

preprint2019arXiv

An Unusual Mid-Infrared Flare in a Type 2 AGN: An Obscured Turning-on AGN or Tidal Disruption Event?

We report the discovery of an exceptional MIR flare in a Type 2 AGN, SDSS J165726.81+234528.1, at $z=0.059$. This object brightened by 3 mag in the Wide-field Infrared Survey Explorer (WISE) $W1$ and $W2$ bands between 2015 and 2017 (and is fading since 2018), without significant changes ($\lesssim$ 0.2 mag) in the optical over the same period of time. Based on the WISE light curves and near-IR imaging, the flare is more significant at longer wavelengths, suggesting an origin of hot dust emission. The estimated black hole mass ($\sim 10^{6.5}\,M_\odot$) from different methods places its peak bolometric luminosity around the Eddington limit. The high luminosity of the MIR flare and its multi-year timescale suggest that it most likely originated from reprocessed dust radiation in an extended torus surrounding the AGN, instead of from stellar explosions. The MIR color variability is consistent with known changing-look AGN and tidal disruption events (TDEs), but inconsistent with normal supernovae. We suggest that it is a turning-on Type 2 AGN or TDE, where the optical variability is obscured by the dust torus during the transition. This MIR flare event reveals a population of dramatic nuclear transients that are missed in the optical.

preprint2019arXiv

Calculation of the $B\to K_{0,2}^*(1430)f_0(980)/σ$ decays in the Perturbative QCD Approach

Motivated by the observations of the decays $B^0 \to K_0^{*}(1430)^0 f_0(980)$ and $ B^0 \to K_2^{*}(1430)^0 f_0(980)$ from BaBar collaboration, we study the $B^{0(+)} \to K_{0,2}^{*}(1430)^{0(+)} f_0(980)/σ$ decays in the perturbative QCD approach for the first time. In the absence of reliable nonperturbative wave functions we only assume the scalar meson $f_0(980)$ and $σ$ are two-quark ground states. In our calculations, these decays are all dominated by the hard-scattering emission and annihilation diagrams, while the factorizable emission diagrams are forbidden or suppressed heavily by the vector decay constants. Furthermore, the branching fractions are sensitive to the mixing between $f_0(980)$ and $σ$. Comparing our results with the experimental data, a large mixing angle $θ$ is favored. Taking $θ=145^\circ$, the orders of branching fractions of $B \to K_0^{*}(1430)^0 σ$, $B \to K_{2}^{*}(1430)^0 σ$ and $B \to K_{0,2}^{*}(1430)^0 f_0(980)$ are predicted to be $10^{-4}$, $10^{-5}$ and $10^{-6}$, respectively, which can be measured in the current experiments such as LHCb and Belle-2. In addition, although these decays are penguin dominant, the mixing also leads to large direct $CP$ asymmetries in these decays. With the precise data in future, our results could shed light on the inner structure of the scalar mesons and can be used to determine the mixing angle of the $σ-f_0(980)$ system.

preprint2019arXiv

Large-scale Mobile App Identification Using Deep Learning

Many network services and tools (e.g. network monitors, malware-detection systems, routing and billing policy enforcement modules in ISPs) depend on identifying the type of traffic that passes through the network. With the widespread use of mobile devices, the vast diversity of mobile apps, and the massive adoption of encryption protocols (such as TLS), large-scale encrypted traffic classification becomes increasingly difficult. In this paper, we propose a deep learning model for mobile app identification that works even with encrypted traffic. The proposed model only needs the payload of the first few packets for classification, and, hence, it is suitable even for applications that rely on early prediction, such as routing and QoS provisioning. The deep model achieves between 84% to 98% accuracy for the identification of 80 popular apps. We also perform occlusion analysis to bring insight into what data is leaked from SSL/TLS protocol that allows accurate app identification. Moreover, our traffic analysis shows that many apps generate not only app-specific traffic, but also numerous ambiguous flows. Ambiguous flows are flows generated by common functionality modules, such as advertisement and traffic analytics. Because such flows are common among many different apps, identifying the source app that generates ambiguous flows is challenging. To address this challenge, we propose a CNN+LSTM model that uses adjacent flows to learn the order and pattern of multiple flows, to better identify the app that generates them. We show that such flow association considerably improves the accuracy, particularly for ambiguous flows. Furthermore, we show that our approach is robust to mixed traffic scenarios where some unrelated flows may appear in adjacent flows. To the best of our knowledge, this is the first work that identifies the source app for ambiguous flows.

preprint2019arXiv

Spectral Energy Distributions of Candidate Periodically-Variable Quasars: Testing the Binary Black Hole Hypothesis

Periodic quasars are candidates for binary supermassive black holes (BSBHs) efficiently emitting low frequency gravitational waves. Recently, $\sim$150 candidates were identified from optical synoptic surveys. However, they may be false positives caused by stochastic quasar variability given the few cycles covered (typically 1.5). To independently test the binary hypothesis, we search for evidence of truncated or gapped circumbinary accretion discs (CBDs) in their spectral energy distributions (SEDs). Our work is motivated by CBD simulations that predict flux deficits as cutoffs from central cavities opened by secondaries or notches from minidiscs around both BHs. We find that candidate periodic quasars show SEDs similar to those of control quasars matched in redshift and luminosity. While seven of 138 candidates show a blue cutoff in the IR-optical-UV SED, six of which may represent CBDs with central cavities, the red SED fraction is similar to that in control quasars, suggesting no correlation between periodicity and SED anomaly. Alternatively, dust reddening may cause red SEDs. The fraction of extremely radio-loud quasars, e.g., blazars (with $R>100$), is tentatively higher than that in control quasars (at 2.5$σ$). Our results suggest that, assuming most periodic candidates are robust, IR-optical-UV SEDs of CBDs are similar to those of accretion discs of single BHs, if the periodicity is driven by BSBHs; the higher blazar fraction may signal precessing radio jets. Alternatively, most current candidate periodic quasars identified from few-cycle light curves may be false positives. Their tentatively higher blazar fraction and lower Eddington ratios may both be caused by selection biases.

preprint2019arXiv

Spectral Variability of a Sample of Extreme Variability Quasars and Implications for the MgII Broad-line Region

We present new Gemini/GMOS optical spectroscopy of 16 extreme variability quasars (EVQs) that dimmed by more than 1.5 mag in the $g$ band between the Sloan Digital Sky Survey (SDSS) and the Dark Energy Survey (DES) epochs (separated by a few years in the quasar rest frame). The quasar sample covers a redshift range of $0.5 < z < 2.1$. Nearly half of these EVQs brightened significantly (by more than 0.5 mag in the $g$ band) in a few years after reaching their previous faintest state, and some EVQs showed rapid (non-blazar) variations of greater than 1-2 mag on timescales of only months. Leveraging on the large dynamic range in continuum variability between the earlier SDSS and the new GMOS spectra, we explore the associated variations in the broad Mg II,$\lambda2798$ line, whose variability properties have not been well studied before. The broad Mg II flux varies in the same direction as the continuum flux, albeit with a smaller amplitude, which indicates at least some portion of Mg II is reverberating to continuum changes. However, the width (FWHM) of Mg II does not vary accordingly as continuum changes for most objects in the sample, in contrast to the case of the broad Balmer lines. Using the width of broad Mg II to estimate the black hole mass therefore introduces a luminosity-dependent bias.

preprint2019arXiv

Super-Resolved Image Perceptual Quality Improvement via Multi-Feature Discriminators

Generative adversarial network (GAN) for image super-resolution (SR) has attracted enormous interests in recent years. However, the GAN-based SR methods only use image discriminator to distinguish SR images and high-resolution (HR) images. Image discriminator fails to discriminate images accurately since image features cannot be fully expressed. In this paper, we design a new GAN-based SR framework GAN-IMC which includes generator, image discriminator, morphological component discriminator and color discriminator. The combination of multiple feature discriminators improves the accuracy of image discrimination. Adversarial training between the generator and multi-feature discriminators forces SR images to converge with HR images in terms of data and features distribution. Moreover, in some cases, feature enhancement of salient regions is also worth considering. GAN-IMC is further optimized by weighted content loss (GAN-IMCW), which effectively restores and enhances salient regions in SR images. The effectiveness and robustness of our method are confirmed by extensive experiments on public datasets. Compared with state-of-the-art methods, the proposed method not only achieves competitive Perceptual Index (PI) and Natural Image Quality Evaluator (NIQE) values but also obtains pleasant visual perception in image edge, texture, color and salient regions.

preprint2019arXiv

Understanding Broad Mg II Variability in Quasars with Photoionization: Implications for Reverberation Mapping and Changing-Look Quasars

The broad Mg II line in quasars has distinct variability properties compared with broad Balmer lines: it is less variable, and usually does not display a &#34;breathing&#34; mode, the increase in the average cloud distance when luminosity increases. We demonstrate that these variability properties of Mg II can be reasonably well explained by simple Locally Optimally Emitting Cloud (LOC) photoionization models, confirming earlier photoionization results. In the fiducial LOC model, the Mg II-emitting gas is on average more distant from the ionizing source than the H$α$/H$β$ gas, and responds with a lower amplitude to continuum variations. If the broad-line region (BLR) is truncated at a physical radius of $\sim 0.3$ pc (for a $10^{8.5}M_{\odot}$ BH accreting at Eddington ratio of 0.1), most of the Mg II flux will always be emitted near this outer boundary and hence will not display breathing. These results indicate that reverberation mapping results on broad Mg II, while generally more difficult to obtain due to the lower line responsivity, can still be used to infer the Mg II BLR size and hence black hole mass. But it is possible that Mg II does not have a well defined intrinsic BLR size-luminosity relation for individual quasars, even though a global one for the general population may still exist. The dramatic changes in broad H$α$/H$β$ emission in the observationally-rare changing-look quasars are fully consistent with photoionization responses to extreme continuum variability, and the LOC model provides natural explanations for the persistence of broad Mg II in changing-look quasars defined on H$α$/H$β$, and the rare population of broad Mg II emitters in the spectra of massive inactive galaxies.

preprint2019arXiv

Varstrometry for Off-nucleus and Dual sub-Kpc AGN (VODKA): Methodology and Initial Results with Gaia DR2

Gaia&#39;s milli-arcsec (mas) astrometric precision allows systematic identification of optically-selected sub-kpc dual active galactic nuclei (AGN), off-nucleus AGN, and small-scale lensed quasars by `varstrometry&#39; -- where variability-induced astrometric jitter, i.e., temporal displacements of photocenter in unresolved sources, can be reasonably well detected or constrained. This approach extends systematic searches for small-scale ($\gtrsim$ mas) dual and off-nucleus AGN to poorly explored regime between $\sim 10$ pc and $\sim 1$ kpc, with Gaia&#39;s full sky coverage and depth to $G\sim 21$. We outline the general principles of this method and calculate the expected astrometric signals from the full time series of photocenter measurements and light curves. We demonstrate the feasibility of varstrometry by using Gaia DR2 data on a sample of variable pre-main sequence stars with known close companions. We find that extended host galaxies have a significant impact on the accuracy of astrometric and photometric variability in Gaia DR2, a situation to be improved in future Gaia releases. Using spectroscopically confirmed SDSS quasars, we present several examples of candidate sub-kpc off-nucleus or dual AGN selected from Gaia DR2. We discuss the merits and limitations of this method and follow-up strategy for promising candidates. We highlight Gaia&#39;s potential of systematically discovering and characterizing the sub-kpc off-nucleus and dual AGN population in the entire optical sky.

preprint2019arXiv

Well-posedness of strong solutions to the anelastic equations of stratified viscous flows

We establish the local and global well-posedness of strong solutions to the two- and three-dimensional anelastic equations of stratified viscous flows. In this model, the interaction of the density profile with the velocity field is taken into account, and the density background profile is permitted to have physical vacuum singularity. The existing time of the solutions is infinite in two dimensions, with general initial data, and in three dimensions with small initial data.

preprint2019arXiv

Zero Mach Number Limit of the Compressible Primitive Equations Part I: Well-prepared Initial Data

This work concerns the zero Mach number limit of the compressible primitive equations. The primitive equations with the incompressibility condition are identified as the limiting equations. The convergence with well-prepared initial data (i.e., initial data without acoustic oscillations) is rigorously justified, and the convergence rate is shown to be of order $ \mathcal O(\varepsilon) $, as $ \varepsilon \rightarrow 0^+ $, where $ \varepsilon $ represents the Mach number. As a byproduct, we construct a class of global solutions to the compressible primitive equations, which are close to the incompressible flows.