Researcher profile

Lei Wang

Lei Wang contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate

Trust snapshot

Quick read

Trust 21 - EmergingVerification L1Unclaimed author

112works

0followers

54topics

4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2026arXiv

Delta Score Matters! Spatial Adaptive Multi Guidance in Diffusion Models

Diffusion models have achieved remarkable success in synthesizing complex static and temporal visuals, a breakthrough largely driven by Classifier-Free Guidance (CFG). However, despite its pivotal role in aligning generated content with textual prompts, standard CFG relies on a globally uniform scalar. This homogeneous amplification traps models in a well-documented "detail-artifact dilemma": low guidance scales fail to inject intricate semantics, while high scales inevitably cause structural degradation, color over-saturation, and temporal inconsistencies in videos. In this paper, we expose the physical root of this flaw through the lens of differential geometry. By analyzing Tweedie's Formula, we reveal that CFG intrinsically performs a tangential linear extrapolation. Because the natural data manifold is highly curved, this uniform linear step introduces a severe orthogonal deviation. To keep the generation trajectory safely bounded, we formulate a theoretical upper bound for spatial and adaptive guidance. Based on these geometric insights, we propose Spatial Adaptive Multi Guidance (SAMG), a training-free and virtually zero-cost sampling algorithm. SAMG dynamically computes point-wise conditional guidance energy, applying a conservative minimum scale to high-energy boundary regions to preserve delicate micro-textures, while deploying an aggressive maximum scale in low-energy regions to maximize semantic injection. Extensive experiments across diverse image (SD 1.5, SDXL, SD3.5 Medium) and video (CogVideoX, ModelScope) architectures demonstrate that SAMG effectively resolves the detail-artifact dilemma, achieving superior semantic alignment, structural integrity, and temporal smoothness without any computational overhead.

preprint2026arXiv

VulTriage: Triple-Path Context Augmentation for LLM-Based Vulnerability Detection

Automated vulnerability detection is a fundamental task in software security, yet existing learning-based methods still struggle to capture the structural dependencies, domain-specific vulnerability knowledge, and complex program semantics required for accurate detection. Recent Large Language Models (LLMs) have shown strong code understanding ability, but directly prompting them with raw source code often leads to missed vulnerabilities or false alarms, especially when vulnerable and benign functions differ only in subtle semantic details. To address this, we propose VulTriage, a triple-path context augmentation framework for LLM-based vulnerability detection. VulTriage enhances the LLM input through three complementary paths: a Control Path that extracts and verbalizes AST, CFG, and DFG information to expose control and data dependencies; a Knowledge Path that retrieves relevant CWE-derived vulnerability patterns and examples through hybrid dense--sparse retrieval; and a Semantic Path that summarizes the functional behavior of the code before the final judgment. These contexts are integrated into a unified instruction to guide the LLM toward more reliable vulnerability reasoning. Experiments on the PrimeVul pair test set show that VulTriage achieves state-of-the-art performance, outperforming existing deep learning and LLM-based baselines on key pair-wise and classification metrics. Further ablation studies verify the effectiveness of each path, and additional experiments on the Kotlin dataset demonstrate the generalization ability of VulTriage under low-resource and class-imbalanced settings. Our code is available at https://github.com/vinsontang1/VulTriage

preprint2025arXiv

Modern applications of machine learning in quantum sciences

In this book, we provide a comprehensive introduction to the most recent advances in the application of machine learning methods in quantum sciences. We cover the use of deep learning and kernel methods in supervised, unsupervised, and reinforcement learning algorithms for phase classification, representation of many-body quantum states, quantum feedback control, and quantum circuits optimization. Moreover, we introduce and discuss more specialized topics such as differentiable programming, generative models, statistical approach to machine learning, and quantum machine learning.

preprint2024arXiv

An Inexact Preconditioned Zeroth-order Proximal Method for Composite Optimization

In this paper, we consider the composite optimization problem, where the objective function integrates a continuously differentiable loss function with a nonsmooth regularization term. Moreover, only the function values for the differentiable part of the objective function are available. To efficiently solve this composite optimization problem, we propose a preconditioned zeroth-order proximal gradient method in which the gradients and preconditioners are estimated by finite-difference schemes based on the function values at the same trial points. We establish the global convergence and worst-case complexity for our proposed method. Numerical experiments exhibit the superiority of our developed method.

preprint2024arXiv

An Initial Investigation of Neural Replay Simulator for Over-the-Air Adversarial Perturbations to Automatic Speaker Verification

Deep Learning has advanced Automatic Speaker Verification (ASV) in the past few years. Although it is known that deep learning-based ASV systems are vulnerable to adversarial examples in digital access, there are few studies on adversarial attacks in the context of physical access, where a replay process (i.e., over the air) is involved. An over-the-air attack involves a loudspeaker, a microphone, and a replaying environment that impacts the movement of the sound wave. Our initial experiment confirms that the replay process impacts the effectiveness of the over-the-air attack performance. This study performs an initial investigation towards utilizing a neural replay simulator to improve over-the-air adversarial attack robustness. This is achieved by using a neural waveform synthesizer to simulate the replay process when estimating the adversarial perturbations. Experiments conducted on the ASVspoof2019 dataset confirm that the neural replay simulator can considerably increase the success rates of over-the-air adversarial attacks. This raises the concern for adversarial attacks on speaker verification in physical access applications.

preprint2024arXiv

Broadband miniaturized spectrometers with a van der Waals tunnel diode

Miniaturized spectrometers are of immense interest for various on-chip and implantable photonic and optoelectronic applications. State-of-the-art conventional spectrometer designs rely heavily on bulky dispersive components (such as gratings, photodetector arrays, and interferometric optics) to capture different input spectral components that increase their integration complexity. Here, we report a high-performance broadband spectrometer based on a simple and compact van der Waals heterostructure diode, leveraging a careful selection of active van der Waals materials -- molybdenum disulfide and black phosphorus, their electrically tunable photoresponse, and advanced computational algorithms for spectral reconstruction. We achieve remarkably high peak wavelength accuracy of ~2 nanometers, and broad operation bandwidth spanning from ~500 to 1600 nanometers in a device with a ~30x20 μm2 footprint. This diode-based spectrometer scheme with broadband operation offers an attractive pathway for various applications, such as sensing, surveillance and spectral imaging.

preprint2023arXiv

A GOA-Based Fault-Tolerant Trajectory Tracking Control for an Underwater Vehicle of Multi-Thruster System without Actuator Saturation

This paper proposes an intelligent fault-tolerant control (FTC) strategy to tackle the trajectory tracking problem of an underwater vehicle (UV) under thruster damage (power loss) cases and meanwhile resolve the actuator saturation brought by the vehicle's physical constraints. In the proposed control strategy, the trajectory tracking component is formed by a refined backstepping algorithm that controls the velocity variation and a sliding mode control deducts the torque/force outputs; the fault-tolerant component is established based on a Grasshopper Optimization Algorithm (GOA), which provides fast convergence speed as well as satisfactory accuracy of deducting optimized reallocation of the thruster forces to compensate for the power loss in different fault cases. Simulations with or without environmental perturbations under different fault cases and comparisons to other traditional FTCs are presented, thus verifying the effectiveness and robustness of the proposed GOA-based fault-tolerant trajectory tracking design.

preprint2023arXiv

Effect of temperature-dependent thermophysical properties on turbulent forced convection under constant heat flux boundary condition

In this study, we performed highly resolved large-eddy simulations (LES) to investigate the influence of variable properties on the forced turbulent convection in a channel. The constant heat flux boundary condition permits wall temperature fluctuations and thus induces variations of fluid properties. Since the effect of viscosity on the flow exhibits $Re_τ^{-1}$ scaling, we only considered $Re_τ= 180$ in the present study. Compared to the flow with constant properties, results indicate that the variable properties have trivial effects on the mean velocity and temperature profiles, Reynolds shear stress, wall-normal heat flux, as well as the small-scale turbulence characteristics. However, we also observed that the turbulence intensities, low-speed streaks, burst motions, and budgets for temperature variance and wall-normal heat flux are modified by the variable properties in a perceptible way. In addition, we showed that the classic wall scaling is a good choice for flow with small and moderate variations of fluid properties.

preprint2023arXiv

Smoothing Gradient Tracking for Decentralized Optimization over the Stiefel Manifold with Non-smooth Regularizers

Recently, decentralized optimization over the Stiefel manifold has attacked tremendous attentions due to its wide range of applications in various fields. Existing methods rely on the gradients to update variables, which are not applicable to the objective functions with non-smooth regularizers, such as sparse PCA. In this paper, to the best of our knowledge, we propose the first decentralized algorithm for non-smooth optimization over Stiefel manifolds. Our algorithm approximates the non-smooth part of objective function by its Moreau envelope, and then existing algorithms for smooth optimization can be deployed. We establish the convergence guarantee with the iteration complexity of $\mathcal{O} (ε^{-4})$. Numerical experiments conducted under the decentralized setting demonstrate the effectiveness and efficiency of our algorithm.

preprint2022arXiv

A Communication-Efficient and Privacy-Aware Distributed Algorithm for Sparse PCA

Sparse principal component analysis (PCA) improves interpretability of the classic PCA by introducing sparsity into the dimension-reduction process. Optimization models for sparse PCA, however, are generally non-convex, non-smooth and more difficult to solve, especially on large-scale datasets requiring distributed computation over a wide network. In this paper, we develop a distributed and centralized algorithm called DSSAL1 for sparse PCA that aims to achieve low communication overheads by adapting a newly proposed subspace-splitting strategy to accelerate convergence. Theoretically, convergence to stationary points is established for DSSAL1. Extensive numerical results show that DSSAL1 requires far fewer rounds of communication than state-of-the-art peer methods. In addition, we make the case that since messages exchanged in DSSAL1 are well-masked, the possibility of private-data leakage in DSSAL1 is much lower than in some other distributed algorithms.

preprint2022arXiv

A joint explanation of W-mass and muon g-2 in 2HDM

Since both $W$-mass and muon $g-2$ can be affected by the mass splittings among extra Higgs bosons $(H,~A,~H^\pm)$ in a 2HDM, we take a model with $μ$-$τ$ LFV interactions to examine the two anomalies reported respectively by CDF II and FNAL. We obtain the following observations: (i) Combined with theoretical constraints, the CDF $W$-mass measurement disfavors $H$ or $A$ to degenerate in mass with $H^\pm$, but allows $H$ and $A$ to degenerate. The mass splitting between $H^\pm$ and $H/A$ is required to be larger than 10 GeV. The $m_{H^\pm}$ and $m_{A}$ are favored to be smaller than 650 GeV for $m_H<120$ GeV, and allowed to have more large values with increasing of $m_H$. (ii) After imposing other relevant experimental constraints, there are parameter spaces that simultaneously satisfy (at $2σ$ level) the CDF $W$-mass, the FNAL muon $g-2$ and the data of lepton universality in $τ$ decays, but the mass splittings among extra Higgs bosons are strictly constrained.

preprint2022arXiv

A Medical Semantic-Assisted Transformer for Radiographic Report Generation

Automated radiographic report generation is a challenging cross-domain task that aims to automatically generate accurate and semantic-coherence reports to describe medical images. Despite the recent progress in this field, there are still many challenges at least in the following aspects. First, radiographic images are very similar to each other, and thus it is difficult to capture the fine-grained visual differences using CNN as the visual feature extractor like many existing methods. Further, semantic information has been widely applied to boost the performance of generation tasks (e.g. image captioning), but existing methods often fail to provide effective medical semantic features. Toward solving those problems, in this paper, we propose a memory-augmented sparse attention block utilizing bilinear pooling to capture the higher-order interactions between the input fine-grained image features while producing sparse attention. Moreover, we introduce a novel Medical Concepts Generation Network (MCGN) to predict fine-grained semantic concepts and incorporate them into the report generation process as guidance. Our proposed method shows promising performance on the recently released largest benchmark MIMIC-CXR. It outperforms multiple state-of-the-art methods in image captioning and medical report generation.

preprint2022arXiv

A New High Energy Efficiency Scheme Based on Two-Dimension Resource Blocks in Wireless Communication Systems

Energy efficiency (EE) plays a key role in future wireless communication network and it is easily to achieve high EE performance in low SNR regime. In this paper, a new high EE scheme is proposed for a MIMO wireless communication system working in the low SNR regime by using two dimension resource allocation. First, we define the high EE area based on the relationship between the transmission power and the SNR. To meet the constraint of the high EE area, both frequency and space dimension are needed. Besides analysing them separately, we decided to consider frequency and space dimensions as a unit and proposed a two-dimension scheme. Furthermore, considering communication in the high EE area may cause decline of the communication quality, we add quality-of-service(QoS) constraint into the consideration and derive the corresponding EE performance based on the effective capacity. We also derive an approximate expression to simplify the complex EE performance. Finally, our numerical results demonstrate the effectiveness of the proposed scheme.

preprint2022arXiv

A Novel Mix-normalization Method for Generalizable Multi-source Person Re-identification

Person re-identification (Re-ID) has achieved great success in the supervised scenario. However, it is difficult to directly transfer the supervised model to arbitrary unseen domains due to the model overfitting to the seen source domains. In this paper, we aim to tackle the generalizable multi-source person Re-ID task (i.e., there are multiple available source domains, and the testing domain is unseen during training) from the data augmentation perspective, thus we put forward a novel method, termed MixNorm, which consists of domain-aware mix-normalization (DMN) and domain-ware center regularization (DCR). Different from the conventional data augmentation, the proposed domain-aware mix-normalization to enhance the diversity of features during training from the normalization view of the neural network, which can effectively alleviate the model overfitting to the source domains, so as to boost the generalization capability of the model in the unseen domain. To better learn the domain-invariant model, we further develop the domain-aware center regularization to better map the produced diverse features into the same space. Extensive experiments on multiple benchmark datasets validate the effectiveness of the proposed method and show that the proposed method can outperform the state-of-the-art methods. Besides, further analysis also reveals the superiority of the proposed method.

preprint2022arXiv

A Variance-Reduced Stochastic Gradient Tracking Algorithm for Decentralized Optimization with Orthogonality Constraints

Decentralized optimization with orthogonality constraints is found widely in scientific computing and data science. Since the orthogonality constraints are nonconvex, it is quite challenging to design efficient algorithms. Existing approaches leverage the geometric tools from Riemannian optimization to solve this problem at the cost of high sample and communication complexities. To relieve this difficulty, based on two novel techniques that can waive the orthogonality constraints, we propose a variance-reduced stochastic gradient tracking (VRSGT) algorithm with the convergence rate of $O(1 / k)$ to a stationary point. To the best of our knowledge, VRSGT is the first algorithm for decentralized optimization with orthogonality constraints that reduces both sampling and communication complexities simultaneously. In the numerical experiments, VRSGT has a promising performance in a real-world autonomous driving application.

preprint2022arXiv

Ab-initio study of interacting fermions at finite temperature with neural canonical transformation

We present a variational density matrix approach to the thermal properties of interacting fermions in the continuum. The variational density matrix is parametrized by a permutation equivariant many-body unitary transformation together with a discrete probabilistic model. The unitary transformation is implemented as a quantum counterpart of neural canonical transformation, which incorporates correlation effects via a flow of fermion coordinates. As the first application, we study electrons in a two-dimensional quantum dot with an interaction-induced crossover from Fermi liquid to Wigner molecule. The present approach provides accurate results in the low-temperature regime, where conventional quantum Monte Carlo methods face severe difficulties due to the fermion sign problem. The approach is general and flexible for further extensions, thus holds the promise to deliver new physical results on strongly correlated fermions in the context of ultracold quantum gases, condensed matter, and warm dense matter physics.

preprint2022arXiv

Active and Passive Hybrid Detection Method for Power CPS False Data Injection Attacks with Improved AKF and GRU-CNN

Influenced by deep penetration of the new generation of information technology, power systems have gradually evolved into highly coupled cyber-physical systems (CPS). Among many possible power CPS network attacks, a false data injection attacks (FDIAs) is the most serious. Taking account of the fact that the existing knowledge-driven detection process for FDIAs has been in a passive detection state for a long time and ignores the advantages of data-driven active capture of features, an active and passive hybrid detection method for power CPS FDIAs with improved adaptive Kalman filter (AKF) and convolutional neural networks (CNN) is proposed in this paper. First, we analyze the shortcomings of the traditional AKF algorithm in terms of filtering divergence and calculation speed. The state estimation algorithm based on non-negative positive-definite adaptive Kalman filter (NDAKF) is improved, and a passive detection method of FDIAs is constructed, with similarity Euclidean distance detection and residual detection at its core. Then, combined with the advantages of gate recurrent unit (GRU) and CNN in terms of temporal memory and feature-expression ability, an active detection method of FDIAs based on a GRU-CNN hybrid neural network is proposed. Finally, the results of joint knowledge-driven and data-driven parallel detection are used to define a mixed fixed-calculation formula, and an active and passive hybrid detection method of FDIAs is established, considering the characteristic constraints of the parallel mode. A simulation system example of power CPS FDIAs verifies the effectiveness and accuracy of the method proposed in this paper.

preprint2022arXiv

An efficient thermal lattice Boltzmann method for simulating three-dimensional liquid-vapor phase change

In this paper, a multiple-relaxation-time lattice Boltzmann (LB) approach is developed for the simulation of three-dimensional (3D) liquid-vapor phase change based on the pseudopotential model. In contrast to some existing 3D thermal LB models for liquid-vapor phase change, the present approach has two advantages: for one thing, the current approach does not require calculating the gradient of volumetric heat capacity [i.e., $\nabla \left( {ρ{c_v}} \right)$], and for another, the current approach is constructed based on the seven discrete velocities in three dimensions (D3Q7), making the current thermal LB model more efficient and easy to implement. Also, based on the scheme proposed by Zhou and He [Phys Fluids 9:1591-1598, 1997], a pressure boundary condition for the D3Q19 lattice is proposed to model the multiphase flow in open systems. The current method is then validated by considering the temperature distribution in a 3D saturated liquid-vapor system, the $d^2$ law and the droplet evaporation on a heated surface. It is observed that the numerical results fit well with the analytical solutions, the results of the finite difference method and the experimental data. Our numerical results indicate that the present approach is reliable and efficient in dealing with the 3D liquid-vapor phase change.

preprint2022arXiv

Attitude estimation from vector measurements: Necessary and sufficient conditions and convergent observer design

The paper addresses the problem of attitude estimation for rigid bodies using (possibly time-varying) vector measurements, for which we provide a necessary and sufficient condition of distinguishability. Such a condition is shown to be strictly weaker than those previously used for attitude observer design. Thereafter, we show that even for the single vector case the resulting condition is sufficient to design almost globally convergent attitude observers, and two explicit designs are obtained. To overcome the weak excitation issue, the first design employs to make full use of historical information, whereas the second scheme dynamically generates a virtual reference vector, which remains non-collinear to the given vector measurement. Simulation results illustrate the accurate estimation despite noisy measurements.

preprint2022arXiv

CenGCN: Centralized Convolutional Networks with Vertex Imbalance for Scale-Free Graphs

Graph Convolutional Networks (GCNs) have achieved impressive performance in a wide variety of areas, attracting considerable attention. The core step of GCNs is the information-passing framework that considers all information from neighbors to the central vertex to be equally important. Such equal importance, however, is inadequate for scale-free networks, where hub vertices propagate more dominant information due to vertex imbalance. In this paper, we propose a novel centrality-based framework named CenGCN to address the inequality of information. This framework first quantifies the similarity between hub vertices and their neighbors by label propagation with hub vertices. Based on this similarity and centrality indices, the framework transforms the graph by increasing or decreasing the weights of edges connecting hub vertices and adding self-connections to vertices. In each non-output layer of the GCN, this framework uses a hub attention mechanism to assign new weights to connected non-hub vertices based on their common information with hub vertices. We present two variants CenGCN\_D and CenGCN\_E, based on degree centrality and eigenvector centrality, respectively. We also conduct comprehensive experiments, including vertex classification, link prediction, vertex clustering, and network visualization. The results demonstrate that the two variants significantly outperform state-of-the-art baselines.

preprint2022arXiv

Contrastive Centroid Supervision Alleviates Domain Shift in Medical Image Classification

Deep learning based medical imaging classification models usually suffer from the domain shift problem, where the classification performance drops when training data and real-world data differ in imaging equipment manufacturer, image acquisition protocol, patient populations, etc. We propose Feature Centroid Contrast Learning (FCCL), which can improve target domain classification performance by extra supervision during training with contrastive loss between instance and class centroid. Compared with current unsupervised domain adaptation and domain generalization methods, FCCL performs better while only requires labeled image data from a single source domain and no target domain. We verify through extensive experiments that FCCL can achieve superior performance on at least three imaging modalities, i.e. fundus photographs, dermatoscopic images, and H & E tissue images.

preprint2022arXiv

Deep Transfer Learning with Graph Neural Network for Sensor-Based Human Activity Recognition

The sensor-based human activity recognition (HAR) in mobile application scenarios is often confronted with sensor modalities variation and annotated data deficiency. Given this observation, we devised a graph-inspired deep learning approach toward the sensor-based HAR tasks, which was further used to build a deep transfer learning model toward giving a tentative solution for these two challenging problems. Specifically, we present a multi-layer residual structure involved graph convolutional neural network (ResGCNN) toward the sensor-based HAR tasks, namely the HAR-ResGCNN approach. Experimental results on the PAMAP2 and mHealth data sets demonstrate that our ResGCNN is effective at capturing the characteristics of actions with comparable results compared to other sensor-based HAR models (with an average accuracy of 98.18% and 99.07%, respectively). More importantly, the deep transfer learning experiments using the ResGCNN model show excellent transferability and few-shot learning performance. The graph-based framework shows good meta-learning ability and is supposed to be a promising solution in sensor-based HAR tasks.

preprint2022arXiv

Dissipation-enabled hydrodynamic conductivity in a tunable bandgap semiconductor

Electronic transport in the regime where carrier-carrier collisions are the dominant scattering mechanism has taken on new relevance with the advent of ultraclean two-dimensional materials. Here we present a combined theoretical and experimental study of ambipolar hydrodynamic transport in bilayer graphene demonstrating that the conductivity is given by the sum of two Drude-like terms that describe relative motion between electrons and holes, and the collective motion of the electron-hole plasma. As predicted, the measured conductivity of gapless, charge-neutral bilayer graphene is sample- and temperature-independent over a wide range. Away from neutrality, the electron-hole conductivity collapses to a single curve, and a set of just four fitting parameters provides quantitative agreement between theory and experiment at all densities, temperatures, and gaps measured. This work validates recent theories for dissipation-enabled hydrodynamic conductivity and creates a link between semiconductor physics and the emerging field of viscous electronics.

preprint2022arXiv

Factorizations of almost simple orthogonal groups of plus type

This is the fifth one in a series of papers classifying the factorizations of almost simple groups with nonsolvable factors. In this paper we deal with orthogonal groups of plus type.

preprint2022arXiv

Fast and Arbitrary Beam Pattern Design for RIS-Assisted Terahertz Wireless Communication

Reconfigurable intelligent surface (RIS) can assist terahertz wireless communication to restore the fragile line-of-sight links and facilitate beam steering. Arbitrary reflection beam patterns are desired to meet diverse requirements in different applications. This paper establishes relationship between RIS beam pattern design with two-dimensional finite impulse response filter design and proposes a fast non-iterative algorithm to solve the problem. Simulations show that the proposed method outperforms baseline method. Hence, it represents a promising solution for fast and arbitrary beam pattern design in RIS-assisted terahertz wireless communication.

preprint2022arXiv

Fusing Higher-order Features in Graph Neural Networks for Skeleton-based Action Recognition

Skeleton sequences are lightweight and compact, and thus are ideal candidates for action recognition on edge devices. Recent skeleton-based action recognition methods extract features from 3D joint coordinates as spatial-temporal cues, using these representations in a graph neural network for feature fusion to boost recognition performance. The use of first- and second-order features, i.e., joint and bone representations, has led to high accuracy. Nonetheless, many models are still confused by actions that have similar motion trajectories. To address these issues, we propose fusing higher-order features in the form of angular encoding into modern architectures to robustly capture the relationships between joints and body parts. This simple fusion with popular spatial-temporal graph neural networks achieves new state-of-the-art accuracy in two large benchmarks, including NTU60 and NTU120, while employing fewer parameters and reduced run time. Our source code is publicly available at: https://github.com/ZhenyueQin/Angular-Skeleton-Encoding.

preprint2022arXiv

Graph Neural Network with Curriculum Learning for Imbalanced Node Classification

Graph Neural Network (GNN) is an emerging technique for graph-based learning tasks such as node classification. In this work, we reveal the vulnerability of GNN to the imbalance of node labels. Traditional solutions for imbalanced classification (e.g. resampling) are ineffective in node classification without considering the graph structure. Worse still, they may even bring overfitting or underfitting results due to lack of sufficient prior knowledge. To solve these problems, we propose a novel graph neural network framework with curriculum learning (GNN-CL) consisting of two modules. For one thing, we hope to acquire certain reliable interpolation nodes and edges through the novel graph-based oversampling based on smoothness and homophily. For another, we combine graph classification loss and metric learning loss which adjust the distance between different nodes associated with minority class in feature space. Inspired by curriculum learning, we dynamically adjust the weights of different modules during training process to achieve better ability of generalization and discrimination. The proposed framework is evaluated via several widely used graph datasets, showing that our proposed model consistently outperforms the existing state-of-the-art methods.

preprint2022arXiv

Hardy-Sobolev inequalities with distance to the boundary weight functions

This is the first part of our research on certain sharp Hardy-Sobolev inequalities and the related elliptic equations. In this part we shall establish some sharp weighted Hardy-Sobolev inequalities whose weights are distance functions to the boundary.

preprint2022arXiv

Indirect Adaptive Control of Nonlinearly Parameterized Nonlinear Dissipative Systems

In this note we address the problem of indirect adaptive (regulation or tracking) control of nonlinear, input affine dissipative systems. It is assumed that the supply rate, the storage and the internal dissipation functions may be expressed as nonlinearly parameterized regression equations where the mappings (depending on the unknown parameters) satisfy a monotonicity condition -- this encompasses a large class of physical systems, including passive systems. We propose to estimate the system parameters using the "power-balance" equation, which is the differential version of the classical dissipation inequality, with a new estimator that ensures global, exponential, parameter convergence under the very weak assumption of interval excitation of the power-balance equation regressor. To design the indirect adaptive controller we make the standard assumption of existence of an asymptotically stabilizing controller that depends -- possibly nonlinearly -- on the unknown plant parameters, and apply a certainty-equivalent control law. The benefits of the proposed approach, with respect to other existing solutions, are illustrated with examples.

preprint2022arXiv

Instance Image Retrieval by Learning Purely From Within the Dataset

Quality feature representation is key to instance image retrieval. To attain it, existing methods usually resort to a deep model pre-trained on benchmark datasets or even fine-tune the model with a task-dependent labelled auxiliary dataset. Although achieving promising results, this approach is restricted by two issues: 1) the domain gap between benchmark datasets and the dataset of a given retrieval task; 2) the required auxiliary dataset cannot be readily obtained. In light of this situation, this work looks into a different approach which has not been well investigated for instance image retrieval previously: {can we learn feature representation \textit{specific to} a given retrieval task in order to achieve excellent retrieval?} Our finding is encouraging. By adding an object proposal generator to generate image regions for self-supervised learning, the investigated approach can successfully learn feature representation specific to a given dataset for retrieval. This representation can be made even more effective by boosting it with image similarity information mined from the dataset. As experimentally validated, such a simple ``self-supervised learning + self-boosting'' approach can well compete with the relevant state-of-the-art retrieval methods. Ablation study is conducted to show the appealing properties of this approach and its limitation on generalisation across datasets.

preprint2022arXiv

Learning Class-Agnostic Pseudo Mask Generation for Box-Supervised Semantic Segmentation

Recently, several weakly supervised learning methods have been devoted to utilize bounding box supervision for training deep semantic segmentation models. Most existing methods usually leverage the generic proposal generators (e.g., dense CRF and MCG) to produce enhanced segmentation masks for further training segmentation models. These proposal generators, however, are generic and not specifically designed for box-supervised semantic segmentation, thereby leaving some leeway for improving segmentation performance. In this paper, we aim at seeking for a more accurate learning-based class-agnostic pseudo mask generator tailored to box-supervised semantic segmentation. To this end, we resort to a pixel-level annotated auxiliary dataset where the class labels are non-overlapped with those of the box-annotated dataset. For learning pseudo mask generator from the auxiliary dataset, we present a bi-level optimization formulation. In particular, the lower subproblem is used to learn box-supervised semantic segmentation, while the upper subproblem is used to learn an optimal class-agnostic pseudo mask generator. The learned pseudo segmentation mask generator can then be deployed to the box-annotated dataset for improving weakly supervised semantic segmentation. Experiments on PASCAL VOC 2012 dataset show that the learned pseudo mask generator is effective in boosting segmentation performance, and our method can further close the performance gap between box-supervised and fully-supervised models. Our code will be made publicly available at https://github.com/Vious/LPG_BBox_Segmentation .

preprint2022arXiv

LibFewShot: A Comprehensive Library for Few-shot Learning

Few-shot learning, especially few-shot image classification, has received increasing attention and witnessed significant advances in recent years. Some recent studies implicitly show that many generic techniques or ``tricks'', such as data augmentation, pre-training, knowledge distillation, and self-supervision, may greatly boost the performance of a few-shot learning method. Moreover, different works may employ different software platforms, backbone architectures and input image sizes, making fair comparisons difficult and practitioners struggle with reproducibility. To address these situations, we propose a comprehensive library for few-shot learning (LibFewShot) by re-implementing eighteen state-of-the-art few-shot learning methods in a unified framework with the same single codebase in PyTorch. Furthermore, based on LibFewShot, we provide comprehensive evaluations on multiple benchmarks with various backbone architectures to evaluate common pitfalls and effects of different training tricks. In addition, with respect to the recent doubts on the necessity of meta- or episodic-training mechanism, our evaluation results confirm that such a mechanism is still necessary especially when combined with pre-training. We hope our work can not only lower the barriers for beginners to enter the area of few-shot learning but also elucidate the effects of nontrivial tricks to facilitate intrinsic research on few-shot learning. The source code is available from https://github.com/RL-VIG/LibFewShot.

preprint2022arXiv

Machine Learning assisted excess noise suppression for continuous-variable quantum key distribution

Excess noise is a major obstacle to high-performance continuous-variable quantum key distribution (CVQKD), which is mainly derived from the amplitude attenuation and phase fluctuation of quantum signals caused by channel instability. Here, an excess noise suppression scheme based on equalization is proposed. In this scheme, the distorted signals can be corrected through equalization assisted by a neural network and pilot tone, relieving the pressure on the post-processing and eliminating the hardware cost. For a free-space channel with more intense fluctuation, a classification algorithm is added to classify the received variables, and then the distinctive equalization correction for different classes is carried out. The experimental results show that the scheme can suppress the excess noise to a lower level, and has a significant performance improvement. Moreover, the scheme also enables the system to cope with strong turbulence. It breaks the bottleneck of long-distance quantum communication and lays a foundation for the large-scale application of CVQKD.

preprint2022arXiv

Machine Learning Based Multimodal Neuroimaging Genomics Dementia Score for Predicting Future Conversion to Alzheimer's Disease

Background: The increasing availability of databases containing both magnetic resonance imaging (MRI) and genetic data allows researchers to utilize multimodal data to better understand the characteristics of dementia of Alzheimer's type (DAT). Objective: The goal of this study was to develop and analyze novel biomarkers that can help predict the development and progression of DAT. Methods: We used feature selection and ensemble learning classifier to develop an image/genotype-based DAT score that represents a subject's likelihood of developing DAT in the future. Three feature types were used: MRI only, genetic only, and combined multimodal data. We used a novel data stratification method to better represent different stages of DAT. Using a pre-defined 0.5 threshold on DAT scores, we predicted whether or not a subject would develop DAT in the future. Results: Our results on Alzheimer's Disease Neuroimaging Initiative (ADNI) database showed that dementia scores using genetic data could better predict future DAT progression for currently normal control subjects (Accuracy=0.857) compared to MRI (Accuracy=0.143), while MRI can better characterize subjects with stable mild cognitive impairment (Accuracy=0.614) compared to genetics (Accuracy=0.356). Combining MRI and genetic data showed improved classification performance in the remaining stratified groups. Conclusion: MRI and genetic data can contribute to DAT prediction in different ways. MRI data reflects anatomical changes in the brain, while genetic data can detect the risk of DAT progression prior to the symptomatic onset. Combining information from multimodal data in the right way can improve prediction performance.

preprint2022arXiv

MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving

Math word problem (MWP) solving faces a dilemma in number representation learning. In order to avoid the number representation issue and reduce the search space of feasible solutions, existing works striving for MWP solving usually replace real numbers with symbolic placeholders to focus on logic reasoning. However, different from common symbolic reasoning tasks like program synthesis and knowledge graph reasoning, MWP solving has extra requirements in numerical reasoning. In other words, instead of the number value itself, it is the reusable numerical property that matters more in numerical reasoning. Therefore, we argue that injecting numerical properties into symbolic placeholders with contextualized representation learning schema can provide a way out of the dilemma in the number representation issue here. In this work, we introduce this idea to the popular pre-training language model (PLM) techniques and build MWP-BERT, an effective contextual number representation PLM. We demonstrate the effectiveness of our MWP-BERT on MWP solving and several MWP-specific understanding tasks on both English and Chinese benchmarks.

preprint2022arXiv

OLxPBench: Real-time, Semantically Consistent, and Domain-specific are Essential in Benchmarking, Designing, and Implementing HTAP Systems

As real-time analysis of the new data become increasingly compelling, more organizations deploy Hybrid Transactional/Analytical Processing (HTAP) systems to support real-time queries on data recently generated by online transaction processing. This paper argues that real-time queries, semantically consistent schema, and domain-specific workloads are essential in benchmarking, designing, and implementing HTAP systems. However, most state-of-the-art and state-of-the-practice benchmarks ignore those critical factors. Hence, they are incommensurable and, at worst, misleading in benchmarking, designing, and implementing HTAP systems. This paper presents OLxPBench, a composite HTAP benchmark suite. OLxPBench proposes: (1) the abstraction of a hybrid transaction, performing a real-time query in-between an online transaction, to model widely-observed behavior pattern -- making a quick decision while consulting real-time analysis; (2) a semantically consistent schema to express the relationships between OLTP and OLAP schema; (3) the combination of domain-specific and general benchmarks to characterize diverse application scenarios with varying resource demands. Our evaluations justify the three design decisions of OLxPBench and pinpoint the bottlenecks of two mainstream distributed HTAP DBMSs. International Open Benchmark Council (BenchCouncil) sets up the OLxPBench homepage at https://www.benchcouncil.org/olxpbench/. Its source code is available from https://github.com/BenchCouncil/olxpbench.git.

preprint2022arXiv

Private, Efficient, and Accurate: Protecting Models Trained by Multi-party Learning with Differential Privacy

Secure multi-party computation-based machine learning, referred to as MPL, has become an important technology to utilize data from multiple parties with privacy preservation. While MPL provides rigorous security guarantees for the computation process, the models trained by MPL are still vulnerable to attacks that solely depend on access to the models. Differential privacy could help to defend against such attacks. However, the accuracy loss brought by differential privacy and the huge communication overhead of secure multi-party computation protocols make it highly challenging to balance the 3-way trade-off between privacy, efficiency, and accuracy. In this paper, we are motivated to resolve the above issue by proposing a solution, referred to as PEA (Private, Efficient, Accurate), which consists of a secure DPSGD protocol and two optimization methods. First, we propose a secure DPSGD protocol to enforce DPSGD in secret sharing-based MPL frameworks. Second, to reduce the accuracy loss led by differential privacy noise and the huge communication overhead of MPL, we propose two optimization methods for the training process of MPL: (1) the data-independent feature extraction method, which aims to simplify the trained model structure; (2) the local data-based global model initialization method, which aims to speed up the convergence of the model training. We implement PEA in two open-source MPL frameworks: TF-Encrypted and Queqiao. The experimental results on various datasets demonstrate the efficiency and effectiveness of PEA. E.g. when $ε$ = 2, we can train a differentially private classification model with an accuracy of 88% for CIFAR-10 within 7 minutes under the LAN setting. This result significantly outperforms the one from CryptGPU, one SOTA MPL framework: it costs more than 16 hours to train a non-private deep neural network model on CIFAR-10 with the same accuracy.

preprint2022arXiv

Progressive Hard-case Mining across Pyramid Levels for Object Detection

In object detection, multi-level prediction (e.g., FPN) and reweighting skills (e.g., focal loss) have drastically improved one-stage detector performance. However, the synergy between these two techniques is not fully explored in a unified framework. We find that, during training, the one-stage detector's optimization is not only restricted to the static hard-case mining loss (gradient drift) but also suffered from the diverse positive samples' proportions split by different pyramid levels (level discrepancy). Under this concern, we propose Hierarchical Progressive Focus (HPF) consisting of two key designs: 1) progressive focus, a more flexible hard-case mining setting calculated adaptive to the convergence progress, 2) hierarchical sampling, automatically generating a set of progressive focus for level-specific target optimization. Based on focal loss with ATSS-R50, our approach achieves 40.5 AP, surpassing the state-of-the-art QFL (Quality Focal Loss, 39.9 AP) and VFL (Varifocal Loss, 40.1 AP). Our best model achieves 55.1 AP on COCO test-dev, obtaining excellent results with only a typical training setting. Moreover, as a plug-and-play scheme, HPF can cooperate well with recent advances, providing a stable performance improvement on nine mainstream detectors.

preprint2022arXiv

Projective-truncation-approximation study of the one-dimensional $ϕ^4$ lattice model

In this paper, we first develop the projective truncation approximation (PTA) in the Green's function equation of motion (EOM) formalism for classical statistical models. To implement PTA for a given Hamiltonian, we choose a set of basis variables and projectively truncate the hierarchical EOM. We apply PTA to the one-dimensional $ϕ^4$ lattice model. Phonon dispersion and static correlation functions are studied in detail. Using one- and two-dimensional bases, we obtain results identical to and beyond the quadratic variational approximation, respectively. In particular, we analyze the power-law temperature dependence of the static averages in the low- and high-temperature limits, and we give exact exponents.

preprint2022arXiv

Pursuing the Precision Study for Color Glass Condensate in Forward Hadron Productions

With the tremendous accomplishments of RHIC and the LHC experiments and the advent of the future Electron-Ion Collider on the horizon, the quest for compelling evidence of the color glass condensate (CGC) has become one of the most aspiring goals in the high energy Quantum Chromodynamics research. Pursuing this question requires developing the precision test of the CGC formalism. By systematically implementing the threshold resummation, we significantly improve the stability of the next-to-leading-order calculation in CGC for forward rapidity hadron productions in $pp$ and $pA$ collisions, especially in the high $p_T$ region, and obtain reliable descriptions of all existing data measured at RHIC and the LHC across all $p_T$ regions. Consequently, this technique can pave the way for the precision studies of the CGC next-to-leading-order predictions by confronting them with a large amount of precise data.

preprint2022arXiv

Revealing the CO2 emission reduction of ridesplitting and its determinants based on real-world data

Ridesplitting, which is a form of pooled ridesourcing service, has great potential to alleviate the negative impacts of ridesourcing on the environment. However, most existing studies only explored its theoretical environmental benefits based on optimization models and simulations. By contrast, this study aims to reveal the real-world emission reduction of ridesplitting and its determinants based on the observed data of ridesourcing in Chengdu, China. Integrating the trip data with the COPERT model, this study calculates the CO2 emissions of shared rides (ridesplitting) and their substituted single rides (regular ridesourcing) to estimate the CO2 emission reduction of each ridesplitting trip. The results show that not all ridesplitting trips reduce emissions from ridesourcing in the real world. The CO2 emission reduction rate of ridesplitting varies from trip to trip, averaging at 43.15g/km. Then, interpretable machine learning models, gradient boosting machines, are applied to explore the relationship between the CO2 emission reduction rate of ridesplitting and its determinants. Based on the SHapley Additive exPlanations (SHAP) method, the overlap rate and detour rate of shared rides are identified to be the most important factors that determine the CO2 emission reduction rate of ridesplitting. Increasing the overlap rate, the number of shared rides, average speed, and ride distance ratio while decreasing the detour rate, actual trip distance, and ride distance gap can increase the CO2 emission reduction rate of ridesplitting. In addition, nonlinear effects and interactions of the determinants are examined through the partial dependence plots. To sum up, this study provides a scientific method for the government and ridesourcing companies to better assess and optimize the environmental benefits of ridesplitting.

preprint2022arXiv

Scalable and Sparsity-Aware Privacy-Preserving K-means Clustering with Application to Fraud Detection

K-means is one of the most widely used clustering models in practice. Due to the problem of data isolation and the requirement for high model performance, how to jointly build practical and secure K-means for multiple parties has become an important topic for many applications in the industry. Existing work on this is mainly of two types. The first type has efficiency advantages, but information leakage raises potential privacy risks. The second type is provable secure but is inefficient and even helpless for the large-scale data sparsity scenario. In this paper, we propose a new framework for efficient sparsity-aware K-means with three characteristics. First, our framework is divided into a data-independent offline phase and a much faster online phase, and the offline phase allows to pre-compute almost all cryptographic operations. Second, we take advantage of the vectorization techniques in both online and offline phases. Third, we adopt a sparse matrix multiplication for the data sparsity scenario to improve efficiency further. We conduct comprehensive experiments on three synthetic datasets and deploy our model in a real-world fraud detection task. Our experimental results show that, compared with the state-of-the-art solution, our model achieves competitive performance in terms of both running time and communication size, especially on sparse datasets.

preprint2022arXiv

Self-consistent Gradient-like Eigen Decomposition in Solving Schrödinger Equations

The Schrödinger equation is at the heart of modern quantum mechanics. Since exact solutions of the ground state are typically intractable, standard approaches approximate Schrödinger equation as forms of nonlinear generalized eigenvalue problems $F(V)V = SVΛ$ in which $F(V)$, the matrix to be decomposed, is a function of its own top-$k$ smallest eigenvectors $V$, leading to a "self-consistency problem". Traditional iterative methods heavily rely on high-quality initial guesses of $V$ generated via domain-specific heuristics methods based on quantum mechanics. In this work, we eliminate such a need for domain-specific heuristics by presenting a novel framework, Self-consistent Gradient-like Eigen Decomposition (SCGLED) that regards $F(V)$ as a special "online data generator", thus allows gradient-like eigendecomposition methods in streaming $k$-PCA to approach the self-consistency of the equation from scratch in an iterative way similar to online learning. With several critical numerical improvements, SCGLED is robust to initial guesses, free of quantum-mechanism-based heuristics designs, and neat in implementation. Our experiments show that it not only can simply replace traditional heuristics-based initial guess methods with large performance advantage (achieved averagely 25x more precise than the best baseline in similar wall time), but also is capable of finding highly precise solutions independently without any traditional iterative methods.

preprint2022arXiv

StyTr$^2$: Image Style Transfer with Transformers

The goal of image style transfer is to render an image with artistic features guided by a style reference while maintaining the original content. Owing to the locality in convolutional neural networks (CNNs), extracting and maintaining the global information of input images is difficult. Therefore, traditional neural style transfer methods face biased content representation. To address this critical issue, we take long-range dependencies of input images into account for image style transfer by proposing a transformer-based approach called StyTr$^2$. In contrast with visual transformers for other vision tasks, StyTr$^2$ contains two different transformer encoders to generate domain-specific sequences for content and style, respectively. Following the encoders, a multi-layer transformer decoder is adopted to stylize the content sequence according to the style sequence. We also analyze the deficiency of existing positional encoding methods and propose the content-aware positional encoding (CAPE), which is scale-invariant and more suitable for image style transfer tasks. Qualitative and quantitative experiments demonstrate the effectiveness of the proposed StyTr$^2$ compared with state-of-the-art CNN-based and flow-based approaches. Code and models are available at https://github.com/diyiiyiii/StyTR-2.

preprint2022arXiv

Testing gravitational redshift based on microwave frequency links onboard China Space Station

In 2022 China Space Station (CSS) will be equipped with atomic clocks and optical clocks with stabilities of $2 \times 10^{-16}$ and $8 \times 10^{-18}$, respectively, which provides an excellent opportunity to test gravitational redshift (GR) with higher accuracy than previous results. Based on high-precise frequency links between CSS and a ground station, we formulated a model and provided simulation experiments to test GR. Simulation results suggest that this method could test the GR at the accuracy level of $(0.27 \pm 2.15) \times10^{-7}$, more than two orders in magnitude higher than the result of the experiment of a hydrogen clock on board a flying rocket more than 40 years ago.

preprint2022arXiv

The Shigesada-Kawasaki-Teramoto cross-diffusion system beyond detailed balance

The existence of global weak solutions to the cross-diffusion model of Shigesada, Kawasaki, and Teramoto for an arbitrary number of species is proved. The model consists of strongly coupled parabolic equations for the population densities in a bounded domain with no-flux boundary conditions, and it describes the dynamics of the segregation of the population species. The diffusion matrix is neither symmetric nor positive semidefinite. A new logarithmic entropy allows for an improved condition on the coefficients of heavily nonsymmetric diffusion matrices, without imposing the detailed-balance condition that is often assumed in the literature. Furthermore, the large-time convergence of the solutions to the constant steady state is proved by using the relative entropy associated to the logarithmic entropy.

preprint2022arXiv

Three-dimensional study of double droplets impact on a wettability-patterned surface

The directional movement and rebound behaviours of two droplets simultaneously impacting a designed flat surface with wettability difference is investigated based on the three-dimensional multi-relaxation-time pseudopotential lattice Boltzmann model. The effects of several factors, such as wettability difference, Weber number and droplet spacing on the directional movement and rebound behaviours are investigated in detail. The numerical results show that the unbalanced Young 's force caused by the wetting difference will cause the droplets to rebound or migrate laterally toward to the side with lower hydrophobicity on the surface, and the contact time of the droplets is found to decrease with the increase of the wetting difference. In addition, it is noted that there exists a secondary spreading behavior in the case of a lower Weber number, which in turn leads to an increase in contact time. Further, as far as the influence of the droplet spacing is concerned, we found that the coalescence intensity of the droplets decreases with the increase of droplet spacing, and in particular, the coalescing droplets are found to divide into two sub-droplets during asymmetric contraction, and three detachment patterns are then defined to reveal the effects of the droplet spacing.

preprint2022arXiv

Topological EEG Nonlinear Dynamics Analysis for Emotion Recognition

Emotional recognition through exploring the electroencephalography (EEG) characteristics has been widely performed in recent studies. Nonlinear analysis and feature extraction methods for understanding the complex dynamical phenomena are associated with the EEG patterns of different emotions. The phase space reconstruction is a typical nonlinear technique to reveal the dynamics of the brain neural system. Recently, the topological data analysis (TDA) scheme has been used to explore the properties of space, which provides a powerful tool to think over the phase space. In this work, we proposed a topological EEG nonlinear dynamics analysis approach using the phase space reconstruction (PSR) technique to convert EEG time series into phase space, and the persistent homology tool explores the topological properties of the phase space. We perform the topological analysis of EEG signals in different rhythm bands to build emotion feature vectors, which shows high distinguishing ability. We evaluate the approach with two well-known benchmark datasets, the DEAP and DREAMER datasets. The recognition results achieved accuracies of 99.37% and 99.35% in arousal and valence classification tasks with DEAP, and 99.96%, 99.93%, and 99.95% in arousal, valence, and dominance classifications tasks with DREAMER, respectively. The performances are supposed to be outperformed current state-of-art approaches in DREAMER (improved by 1% to 10% depends on temporal length), while comparable to other related works evaluated in DEAP. The proposed work is the first investigation in the emotion recognition oriented EEG topological feature analysis, which brought a novel insight into the brain neural system nonlinear dynamics analysis and feature extraction.

preprint2022arXiv

Two-dimensional Obstructed Atomic Insulators with Fractional Corner Charge in MA$_2$Z$_4$ Family

According to topological quantum chemistry, a class of electronic materials have been called obstructed atomic insulators (OAIs), in which a portion of valence electrons necessarily have their centers located on some empty $\textit{Wyckoff}$ positions without atoms occupation in the lattice. The obstruction of centering these electrons coinciding with their host atoms is nontrivial and results in metallic boundary states when the boundary is properly cut. Here, on basis of first-principles calculations in combination with topological quantum chemistry analysis, we propose two dimensional MA$_2$Z$_4$ (M = Cr, Mo and W; A = Si and Ge, Z = N, P and As) monolayer family are all OAIs. A typical case is the recently synthesized MoSi$_2$N$_4$. Although it is a topological trivial insulator with the occupied electronic states being integer combination of elementary band representations, it has valence electrons centering empty $\textit{Wyckoff}$ positions. It exhibits unique OAI-induced metallic edge states along the (1$\bar{1}$0) edge of MoSi$_2$N$_4$ monolayer and the in-gap corner states at three vertices of certain hexagonal nanodisk samples respecting C$_3$ rotation symmetry. The readily synthesized MoSi$_2$N$_4$ is quite stable and has a large bulk band gap of 1.94 eV, which makes the identification of these edge and corner states most possible for experimental clarification.

preprint2022arXiv

Two-stream Hierarchical Similarity Reasoning for Image-text Matching

Reasoning-based approaches have demonstrated their powerful ability for the task of image-text matching. In this work, two issues are addressed for image-text matching. First, for reasoning processing, conventional approaches have no ability to find and use multi-level hierarchical similarity information. To solve this problem, a hierarchical similarity reasoning module is proposed to automatically extract context information, which is then co-exploited with local interaction information for efficient reasoning. Second, previous approaches only consider learning single-stream similarity alignment (i.e., image-to-text level or text-to-image level), which is inadequate to fully use similarity information for image-text matching. To address this issue, a two-stream architecture is developed to decompose image-text matching into image-to-text level and text-to-image level similarity computation. These two issues are investigated by a unifying framework that is trained in an end-to-end manner, namely two-stream hierarchical similarity reasoning network. The extensive experiments performed on the two benchmark datasets of MSCOCO and Flickr30K show the superiority of the proposed approach as compared to existing state-of-the-art methods.

preprint2021arXiv

An efficient HTS electromagnetic model combining thin-strip, homogeneous and multi-scale methods by T-A formulation

This study presents an HTS electromagnetic model combining the thin-strip, homogeneous and multi-scale methods using T-A formulation. In particular, we build the thin strips as both the analyzed HTS tapes and the boundaries of the homogeneous bulks where the non-analyzed tapes are merged. Thus, the coil geometry is re-constructed with several bulks, but the bulks boundaries and domains are tackled with different electromagnetic properties, and solved by T and A formulations, respectively. Firstly, we introduce the modeling process and highlight the differences and advantages over the previous models. Then, the accuracy of the proposed model is validated by comparing the results with those from the reference model based on a 2000-turn coil. The distributions of normalized current density, magnetic flux density and hysteresis losses from the two models are highly consistent, and the error of the total loss is less than 1%. Besides, the proposed model is the most time-saving among all the advanced models. Furthermore, the model can be applied in 3D simulations, and the high accuracy and efficiency are validated by simulating a 50-turn racetrack coil. The proposed method provides a feasible approach to simulating coils with many stacked tapes, and we will continue exploring more applications in solving HTS systems with complex geometries.

preprint2021arXiv

Crosslink-Net: Double-branch Encoder Segmentation Network via Fusing Vertical and Horizontal Convolutions

Accurate image segmentation plays a crucial role in medical image analysis, yet it faces great challenges of various shapes, diverse sizes, and blurry boundaries. To address these difficulties, square kernel-based encoder-decoder architecture has been proposed and widely used, but its performance remains still unsatisfactory. To further cope with these challenges, we present a novel double-branch encoder architecture. Our architecture is inspired by two observations: 1) Since the discrimination of features learned via square convolutional kernels needs to be further improved, we propose to utilize non-square vertical and horizontal convolutional kernels in the double-branch encoder, so features learned by the two branches can be expected to complement each other. 2) Considering that spatial attention can help models to better focus on the target region in a large-sized image, we develop an attention loss to further emphasize the segmentation on small-sized targets. Together, the above two schemes give rise to a novel double-branch encoder segmentation framework for medical image segmentation, namely Crosslink-Net. The experiments validate the effectiveness of our model on four datasets. The code is released at https://github.com/Qianyu1226/Crosslink-Net.

preprint2021arXiv

Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Function

In this paper, we focus on the decentralized optimization problem over the Stiefel manifold, which is defined on a connected network of $d$ agents. The objective is an average of $d$ local functions, and each function is privately held by an agent and encodes its data. The agents can only communicate with their neighbors in a collaborative effort to solve this problem. In existing methods, multiple rounds of communications are required to guarantee the convergence, giving rise to high communication costs. In contrast, this paper proposes a decentralized algorithm, called DESTINY, which only invokes a single round of communications per iteration. DESTINY combines gradient tracking techniques with a novel approximate augmented Lagrangian function. The global convergence to stationary points is rigorously established. Comprehensive numerical experiments demonstrate that DESTINY has a strong potential to deliver a cutting-edge performance in solving a variety of testing problems.

preprint2021arXiv

Differentially Private Distributed Computation via Public-Private Communication Networks

This paper studies the problem of multi-agent computation under the differential privacy requirement of the agents' local datasets against eavesdroppers having node-to-node communications. We first propose for the network equipped with public-private networks. The private network is sparse and not even necessarily connected, over which communications are encrypted and secure along with the intermediate node states; the public network is connected and may be dense, over which communications are allowed to be public. In this setting, we propose a multi-gossip PPSC mechanism over the private network, where at each step, randomly selected node pairs update their states in such a way that they are shuffled with random noise while maintaining summation consistency. We show that this mechanism can achieve any desired differential privacy level with any prescribed probability. Next, we embed this mechanism in distributed computing processes, and propose privacy-guarantee protocols for three basic computation tasks, where an adaptive mechanism adjusts the amount of noise injected in PPSC steps for privacy protection, and the number of regular computation steps for accuracy guarantee. For average consensus, we develop a PPSC-Gossip averaging consensus algorithm by utilizing the multi-gossip PPSC mechanism for privacy encryption before an averaging consensus algorithm over the public network for local computations. For network linear equations and distributed convex optimization, we develop two respective distributed computing protocols by following the PPSC-Gossip averaging consensus algorithm with an additional projection or gradient descent step within each step of computation. Given any privacy and accuracy requirements, it is shown that all three proposed protocols can compute their corresponding problems with the desired computation accuracy, while achieving the desired differential privacy.

preprint2021arXiv

Distributed Algorithms that Solve Boolean Equations with Local and Differential Privacies

In this paper, we propose distributed algorithms that solve a system of Boolean equations over a network, where each node in the network possesses only one Boolean equation from the system. The Boolean equation assigned at any particular node is a {\em private} equation known to this node only, and the nodes aim to compute the exact set of solutions to the system without exchanging their local equations. We show that each private Boolean equation can be locally lifted to a linear algebraic equation under a basis of Boolean vectors, leading to a network linear equation that is distributedly solvable using existing distributed linear equation algorithms as a subroutine. A number of exact or approximate solutions to the induced linear equation are then computed at each node from different initial values. The solutions to the original Boolean equations are eventually computed locally via a Boolean vector search algorithm. We prove that given solvable Boolean equations, when the initial values of the nodes for the distributed linear equation solving step are i.i.d selected according to a uniform distribution in a high-dimensional cube, our algorithms return the exact solution set of the Boolean equations at each node with high probability. Furthermore, we present an algorithm for distributed verification of the satisfiability of Boolean equations, and prove its correctness. Finally, we show that by utilizing linear equation solvers with differential privacy to replace the in-network computing routines, the overall distributed Boolean equation algorithms can be made differentially private. Under the standard Laplace mechanism, we prove an explicit level of noises that can be injected in the linear equation steps for ensuring a prescribed level of differential privacy.

preprint2021arXiv

Fast Evaporation Enabled Ultrathin Polymeric Coatings on Nanoporous Substrates for Highly Permeable Membranes

Membranes derived from ultrathin polymeric films are promising to meet fast separations, but currently available approaches to produce polymer films with greatly reduced thicknesses on porous supports still faces challenges. Here, defect-free ultrathin polymer covering films (UPCFs) are realized by a facile general approach of rapid solvent evaporation. By fast evaporating dilute polymer solutions, we realize ultrathin coating (~30 nm) of porous substrates exclusively on the top surface, forming UPCFs with a block copolymer of polystyrene-block-poly(2-vinyl pyridine) at room temperature or a homopolymer of poly(vinyl alcohol) (PVA) at elevated temperatures. With subsequent selective swelling to the block copolymer and crosslinking to PVA, the resulting bi-layered composite structures serve as highly permeable membranes delivering ~2-10 times higher permeability in ultrafiltration and pervaporation applications than state-of-the-art separation membranes with similar rejections and selectivities. This work opens up a new, facile avenue for the controllable fabrication of ultrathin coatings on porous substrates, which shows great potentials in membrane-based separations and other areas.

preprint2021arXiv

Giant Crystal Hall Effect in Collinear Antiferromagnetic $γ$-FeMn

The spontaneous Hall effect is usually governed by three conventional mechanisms, such as the Berry curvature, skew scattering and side jump, which widely exist in ferromagnetic or antiferromagnetic materials. However, in this work, based on first principle calculations, we predict a giant crystal Hall effect (CHE) in the antiferromagnetic $γ$-FeMn, which can not be understood by the previous three conventional mechanisms and the Hall angle therein can be as large as 18.4% at low temperature. Furthermore, with Boltzmann transport equation and a tight-binding model, we conclude that, the asymmetric group velocities on Fermi surface is the origin of this CHE in $γ$-FeMn. And with a systematic symmetry argument, we show that, this unusual effect is not dependent on specific materials but universal in any crystals with similar symmetry even without local magnetization.

preprint2021arXiv

HPC AI500: Representative, Repeatable and Simple HPC AI Benchmarking

Recent years witness a trend of applying large-scale distributed deep learning algorithms (HPC AI) in both business and scientific computing areas, whose goal is to speed up the training time to achieve a state-of-the-art quality. The HPC AI benchmarks accelerate the process. Unfortunately, benchmarking HPC AI systems at scale raises serious challenges. This paper presents a representative, repeatable and simple HPC AI benchmarking methodology. Among the seventeen AI workloads of AIBench Training -- by far the most comprehensive AI Training benchmarks suite -- we choose two representative and repeatable AI workloads. The selected HPC AI benchmarks include both business and scientific computing: Image Classification and Extreme Weather Analytics. To rank HPC AI systems, we present a new metric named Valid FLOPS, emphasizing both throughput performance and a target quality. The specification, source code, datasets, and HPC AI500 ranking numbers are publicly available from \url{https://www.benchcouncil.org/HPCAI500/}.

preprint2021arXiv

Modeling Method for the Coupling Relations of Microgrid Cyber-Physical Systems Driven by Hybrid Spatiotemporal Events

The essence of the microgrid cyber-physical system (CPS) lies in the cyclical conversion of information flow and energy flow. Most of the existing coupling models are modeled with static networks and interface structures, in which the closed-loop data flow characteristic is not fully considered. It is difficult for these models to accurately describe spatiotemporal deduction processes, such as microgrid CPS attack identification, risk propagation, safety assessment, defense control, and cascading failure. To address this problem, a modeling method for the coupling relations of microgrid CPS driven by hybrid spatiotemporal events is proposed in the present work. First, according to the topological correlation and coupling logic of the microgrid CPS, the cyclical conversion mechanism of information flow and energy flow is analyzed, and a microgrid CPS architecture with multi-agents as the core is constructed. Next, the spatiotemporal evolution characteristic of the CPS is described by hybrid automata, and the task coordination mechanism of the multi-agent CPS terminal is designed. On this basis, a discrete-continuous correlation and terminal structure characteristic representation method of the CPS based on heterogeneous multi-groups are then proposed. Finally, four spatiotemporal events, namely state perception, network communication, intelligent decision-making, and action control, are defined. Considering the constraints of the temporal conversion of information flow and energy flow, a microgrid CPS coupling model is established, the effectiveness of which is verified by simulating false data injection attack (FDIA) scenarios.

preprint2021arXiv

Multiscale analysis of crystal defect formation in rapid solidification of pure aluminium and aluminium-copper alloys

Rapid solidification leads to unique microstructural features, where a less studied topic is the formation of various crystalline defects, including high dislocation densities, as well as gradients and splitting of the crystalline orientation. As these defects critically affect the material's mechanical properties and performance features, it is important to understand the defect formation mechanisms, and how they depend on the solidification conditions and alloying. To illuminate the formation mechanisms of the rapid solidification induced crystalline defects, we conduct a multiscale modeling analysis consisting of bond-order potential based molecular dynamics (MD), phase field crystal based amplitude expansion (PFC-AE) simulations, and sequentially coupled phase field -- crystal plasticity (PF--CP) simulations. The resulting dislocation densities are quantified and compared to past experiments. The atomistic approaches (MD, PFC) can be used to calibrate continuum level crystal plasticity models, and the framework adds mechanistic insights arising from the multiscale analysis.

preprint2021arXiv

Network Representation Learning: From Traditional Feature Learning to Deep Learning

Network representation learning (NRL) is an effective graph analytics technique and promotes users to deeply understand the hidden characteristics of graph data. It has been successfully applied in many real-world tasks related to network science, such as social network data processing, biological information processing, and recommender systems. Deep Learning is a powerful tool to learn data features. However, it is non-trivial to generalize deep learning to graph-structured data since it is different from the regular data such as pictures having spatial information and sounds having temporal information. Recently, researchers proposed many deep learning-based methods in the area of NRL. In this survey, we investigate classical NRL from traditional feature learning method to the deep learning-based model, analyze relationships between them, and summarize the latest progress. Finally, we discuss open issues considering NRL and point out the future directions in this field.

preprint2021arXiv

Novel Two-Dimensional Layered MSi$_2$N$_4$ (M = Mo, W): New Promising Thermal Management Materials

With the miniaturization and integration of nanoelectronic devices, efficient heat removal becomes a key factor affecting the reliable operation of the nanoelectronic device. With the high intrinsic thermal conductivity, good mechanical flexibility, and precisely controlled growth, two-dimensional (2D) materials are widely accepted as ideal candidates for thermal management materials. In this work, by solving the phonon Boltzmann transport equation (BTE) based on first-principles calculations, we comprehensively investigated the thermal conductivity of novel 2D layered MSi$_2$N$_4$ (M = Mo, W). Our results point to competitive thermal conductivities (162 W/mK) of monolayer MoSi$_2$N$_4$, which is around two times larger than that of WSi$_2$N$_4$ and seven times larger than that of silicene despite their similar non-planar structures. It is revealed that the high thermal conductivity arises mainly from its large group velocity and low anharmonicity. Our result suggests that MoSi$_2$N$_4$ could be a potential candidate for 2D thermal management materials.

preprint2021arXiv

Robust I&I Adaptive Tracking Control of Systems with Nonlinear Parameterization: An ISS Perspective

This paper studies the immersion and invariance (I&I) adaptive tracking problem for a class of nonlinear systems with nonlinear parameterization in the ISS framework. Under some mild assumptions, a novel I&I adaptive control algorithm is proposed,leading to an interconnection of an ISS estimation error subsystem and an ISS tracking error subsystem. Using an ISS small-gain condition, the desired uniform global asymptotic stability of the resulting interconnected "error" system can be achieved and a sum-type strict Lyapunov function can be explicitly constructed. Taking advantage of this ISS-based design framework,it is shown that the corresponding robustness with respect to the input perturbation can be rendered to be ISS. To remove the need to solve the immersion manifold shaping PDE, a new filter-based approach is proposed, which preserves the ISS-based design framework. Finally, we demonstrate the validness of the proposed framework on a tracking problem for series elastic actuators.

preprint2021arXiv

Robust Implementable Regulator Design of General Linear Systems

Robust implementable output regulator design approaches are studied for general linear continuous-time \mbox{systems} with periodically sampled measurements, consisting of both the regulation errors and extra measurements that are generally non-vanishing in steady state. A digital regulator is first developed via the conventional emulation-based approach, rendering the regulation errors asymptotically bounded with a small sampling period. We then develop a hybrid design framework by incorporating a generalized hold device, which transforms the original problem into the problem of designing an output feedback controller fulfilling two conditions for a discrete-time system. We show that such a controller can always be obtained by designing a discrete-time internal model, a discrete-time washout filter, and a discrete-time output feedback stabilizer. As a result, the regulation errors are shown to be globally exponentially convergent to zero, while the sampling period is fixed but can be arbitrarily large. This design framework is further developed for a multi-rate digital regulator with a large sampling period of the measurements and a small control execution period.

preprint2021arXiv

Robust Output Feedback Stabilization of MIMO Invertible Nonlinear Systems with Output-Dependent Multipliers (extended version)

This note studies the robust output feedback stabilization problem of multi-input multi-output invertible nonlinear systems with output-dependent multipliers. An "ideal" state feedback is first designed under certain mild assumptions. Then, a set of extended low-power high-gain observers is systematically designed, providing a complete estimation of the "ideal" feedback law. This yields a robust output feedback stabilizer such that the origin of the closed-loop system is semiglobally asymptotically stable, while improving the numerical implementation with the power of high-gain parameters up to 2.

preprint2021arXiv

Robust Output Feedback Stabilization of Multivariable Invertible Nonlinear Systems: A Feedback Linearization-Based Method

This note studies the robust output feedback stabilization problem of a class of multi-input multi-output invertible nonlinear systems, for which an "ideal" state feedback based on feedback linearization can be designed under certain mild assumptions. By systematically designing a set of extended low-power high-gain observers, we show that this "ideal" linearizing feedback law can be approximately estimated, which provides a robust output feedback stabilizer such that the origin of the resulting closed-loop system is semiglobally asymptotically stable.

preprint2021arXiv

Simulation of an imaging system for internal contamination of lungs using MPA-MURA coded aperture collimator

The nuclides inhaled during nuclear accidents usually cause internal contamination of the lungs with low activity. Although a parallel-hole imaging system, which is widely used in medical gamma cameras, has a high resolution and good image quality, owing to its extremely low detection efficiency, it remains difficult to obtain images of inhaled lung contamination. In this study, the Monte Carlo method was used to study the internal lung contamination imaging using the MPA-MURA coded-aperture collimator. The imaging system consisted of an adult male lung model, with a mosaicked, pattern-centered, and anti-symmetric MURA coded-aperture collimator model and a CsI(Tl) detector model. The MLEM decoding algorithm was used to reconstruct the internal contamination image, and the complementary imaging method was used to reduce the number of artifacts. The full width at half maximum of the I-131 point source image reconstructed by the mosaicked, pattern-centered, and anti-symmetric Modified uniformly redundant array (MPA-MURA) coded-aperture imaging reached 2.51 mm, and the signal-to-noise ratio of the simplified respiratory tract source (I-131) image reconstructed through MPA-MURA coded-aperture imaging was 3.98 dB. Although the spatial resolution of MPA-MURA coded aperture imaging is not as good as that of parallel-hole imaging, the detection efficiency of PMA-MURA coded-aperture imaging is two orders of magnitude higher than that of parallel hole collimator imaging. Considering the low activity level of internal lung contamination caused by nuclear accidents, PMA-MURA coded-aperture imaging has significant potential for the development of lung contamination imaging.

preprint2021arXiv

Tropical Tensor Network for Ground States of Spin Glasses

We present a unified exact tensor network approach to compute the ground state energy, identify the optimal configuration, and count the number of solutions for spin glasses. The method is based on tensor networks with the Tropical Algebra defined on the semiring. Contracting the tropical tensor network gives the ground state energy; differentiating through the tensor network contraction gives the ground state configuration; mixing the tropical algebra and the ordinary algebra counts the ground state degeneracy. The approach brings together the concepts from graphical models, tensor networks, differentiable programming, and quantum circuit simulation, and easily utilizes the computational power of graphical processing units (GPUs). For applications, we compute the exact ground state energy of Ising spin glasses on square lattice up to 1024 spins, on cubic lattice up to 216 spins, and on 3 regular random graphs up to 220 spins, on a single GPU; We obtain exact ground state energy of (+/-)J Ising spin glass on the chimera graph of D-Wave quantum annealer of 512 qubits in less than 100 seconds and investigate the exact value of the residual entropy of (+/-)J spin glasses on the chimera graph; Finally, we investigate ground-state energy and entropy of 3-state Potts glasses on square lattices up to size 18 x 18. Our approach provides baselines and benchmarks for exact algorithms for spin glasses and combinatorial optimization problems, and for evaluating heuristic algorithms and mean-field theories.

preprint2021arXiv

Unified First-Principles Study of the Anomalous Hall Effect Based on Exact Muffin-Tin Orbitals

Based on the exact muffin-tin orbitals (EMTOs), we developed a first-principles method to calculate the current operators and investigated the anomalous Hall effect in bcc Fe as an example, with which we successfully separated the skew scattering contribution from the side jump and intrinsic contributions by fitting the scaling law with the introduction of sparse impurities. By investigating the temperature dependence of the anomalous Hall effect in bulk Fe, we predicted a fluctuated anomalous Hall angle as a function of temperature when considering only phonons, which, in the future, can be measured in experiments by suppressing magnon excitation, e.g., by applying a high external magnetic field.

preprint2020arXiv

A Neural Architecture Search based Framework for Liquid State Machine Design

Liquid State Machine (LSM), also known as the recurrent version of Spiking Neural Networks (SNN), has attracted great research interests thanks to its high computational power, biological plausibility from the brain, simple structure and low training complexity. By exploring the design space in network architectures and parameters, recent works have demonstrated great potential for improving the accuracy of LSM model with low complexity. However, these works are based on manually-defined network architectures or predefined parameters. Considering the diversity and uniqueness of brain structure, the design of LSM model should be explored in the largest search space possible. In this paper, we propose a Neural Architecture Search (NAS) based framework to explore both architecture and parameter design space for automatic dataset-oriented LSM model. To handle the exponentially-increased design space, we adopt a three-step search for LSM, including multi-liquid architecture search, variation on the number of neurons and parameters search such as percentage connectivity and excitatory neuron ratio within each liquid. Besides, we propose to use Simulated Annealing (SA) algorithm to implement the three-step heuristic search. Three datasets, including image dataset of MNIST and NMNIST and speech dataset of FSDD, are used to test the effectiveness of our proposed framework. Simulation results show that our proposed framework can produce the dataset-oriented optimal LSM models with high accuracy and low complexity. The best classification accuracy on the three datasets is 93.2%, 92.5% and 84% respectively with only 1000 spiking neurons, and the network connections can be averagely reduced by 61.4% compared with a single LSM. Moreover, we find that the total quantity of neurons in optimal LSM models on three datasets can be further reduced by 20% with only about 0.5% accuracy loss.

preprint2020arXiv

A Noise Filter for Dynamic Vision Sensors using Self-adjusting Threshold

Neuromorphic event-based dynamic vision sensors (DVS) have much faster sampling rates and a higher dynamic range than frame-based imagers. However, they are sensitive to background activity (BA) events which are unwanted. we propose a new criterion with little computation overhead for defining real events and BA events by utilizing the global space and time information rather than the local information by Gaussian convolution, which can be also used as a filter. We denote the filter as GF. We demonstrate GF on three datasets, each recorded by a different DVS with different output size. The experimental results show that our filter produces the clearest frames compared with baseline filters and run fast.

preprint2020arXiv

ASMFS: Adaptive-Similarity-based Multi-modality Feature Selection for Classification of Alzheimer's Disease

With the increasing amounts of high-dimensional heterogeneous data to be processed, multi-modality feature selection has become an important research direction in medical image analysis. Traditional methods usually depict the data structure using fixed and predefined similarity matrix for each modality separately, without considering the potential relationship structure across different modalities. In this paper, we propose a novel multi-modality feature selection method, which performs feature selection and local similarity learning simultaniously. Specially, a similarity matrix is learned by jointly considering different imaging modalities. And at the same time, feature selection is conducted by imposing sparse l_{2, 1} norm constraint. The effectiveness of our proposed joint learning method can be well demonstrated by the experimental results on Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, which outperforms existing the state-of-the-art multi-modality approaches.

preprint2020arXiv

Automatic Differentiation for Second Renormalization of Tensor Networks

Tensor renormalization group (TRG) constitutes an important methodology for accurate simulations of strongly correlated lattice models. Facilitated by the automatic differentiation technique widely used in deep learning, we propose a uniform framework of differentiable TRG ($\partial$TRG) that can be applied to improve various TRG methods, in an automatic fashion. Essentially, $\partial$TRG systematically extends the concept of second renormalization [PRL 103, 160601 (2009)] where the tensor environment is computed recursively in the backward iteration, in the sense that given the forward process of TRG, $\partial$TRG automatically finds the gradient through backpropagation, with which one can deeply "train" the tensor networks. We benchmark $\partial$TRG in solving the square-lattice Ising model, and demonstrate its power by simulating one- and two-dimensional quantum systems at finite temperature. The deep optimization as well as GPU acceleration renders $\partial$TRG manybody simulations with high efficiency and accuracy.

preprint2020arXiv

Automatic differentiation of dominant eigensolver and its applications in quantum physics

We investigate the automatic differentiation of dominant eigensolver where only a small proportion of eigenvalues and corresponding eigenvectors are obtained. Backpropagation through the dominant eigensolver involves solving certain low-rank linear systems without direct access to the full spectrum of the problem. Furthermore, the backward pass can be conveniently differentiated again, which implies that in principle one can obtain arbitrarily higher order derivatives of the dominant eigen-decomposition process. These results allow for the construction of an efficient dominant eigensolver primitive, which has wide applications in quantum physics. As a demonstration, we compute second order derivative of the ground state energy and fidelity susceptibility of 1D transverse field Ising model through the exact diagonalization approach. We also calculate the ground state energy of the same model in the thermodynamic limit by performing gradient-based optimization of uniform matrix product states. By programming these computation tasks in a fully differentiable way, one can efficiently handle the dominant eigen-decomposition of very large matrices while still sharing various advantages of differentiable programming paradigm, notably the generic nature of the implementation and free of tedious human efforts of deriving gradients analytically.

preprint2020arXiv

Chemical-protein Interaction Extraction via Gaussian Probability Distribution and External Biomedical Knowledge

Motivation: The biomedical literature contains a wealth of chemical-protein interactions (CPIs). Automatically extracting CPIs described in biomedical literature is essential for drug discovery, precision medicine, as well as basic biomedical research. Most existing methods focus only on the sentence sequence to identify these CPIs. However, the local structure of sentences and external biomedical knowledge also contain valuable information. Effective use of such information may improve the performance of CPI extraction. Results: In this paper, we propose a novel neural network-based approach to improve CPI extraction. Specifically, the approach first employs BERT to generate high-quality contextual representations of the title sequence, instance sequence, and knowledge sequence. Then, the Gaussian probability distribution is introduced to capture the local structure of the instance. Meanwhile, the attention mechanism is applied to fuse the title information and biomedical knowledge, respectively. Finally, the related representations are concatenated and fed into the softmax function to extract CPIs. We evaluate our proposed model on the CHEMPROT corpus. Our proposed model is superior in performance as compared with other state-of-the-art models. The experimental results show that the Gaussian probability distribution and external knowledge are complementary to each other. Integrating them can effectively improve the CPI extraction performance. Furthermore, the Gaussian probability distribution can effectively improve the extraction performance of sentences with overlapping relations in biomedical relation extraction tasks. Availability: Data and code are available at https://github.com/CongSun-dlut/CPI_extraction. Contact: yangzh@dlut.edu.cn, wangleibihami@gmail.com Supplementary information: Supplementary data are available at Bioinformatics online.

preprint2020arXiv

Class Distribution Alignment for Adversarial Domain Adaptation

Most existing unsupervised domain adaptation methods mainly focused on aligning the marginal distributions of samples between the source and target domains. This setting does not sufficiently consider the class distribution information between the two domains, which could adversely affect the reduction of domain gap. To address this issue, we propose a novel approach called Conditional ADversarial Image Translation (CADIT) to explicitly align the class distributions given samples between the two domains. It integrates a discriminative structure-preserving loss and a joint adversarial generation loss. The former effectively prevents undesired label-flipping during the whole process of image translation, while the latter maintains the joint distribution alignment of images and labels. Furthermore, our approach enforces the classification consistence of target domain images before and after adaptation to aid the classifier training in both domains. Extensive experiments were conducted on multiple benchmark datasets including Digits, Faces, Scenes and Office31, showing that our approach achieved superior classification in the target domain when compared to the state-of-the-art methods. Also, both qualitative and quantitative results well supported our motivation that aligning the class distributions can indeed improve domain adaptation.

preprint2020arXiv

Cm2 Scale Synthesis of MoTe2 Thin Films with Large Grains and Layer Control David

Owing to the small energy differences between its polymorphs, MoTe2 can access a full spectrum of electronic states, from the 2H semiconducting state to the 1T semimetallic state, and from the Td Weyl semimetallic state to the superconducting state in the 1T and Td phase at low temperature. Thus, it is a model system for phase transformation studies as well as quantum phenomena such as the quantum spin Hall effect and topological superconductivity. Careful studies of MoTe2 and its potential applications require large area MoTe2 thin films with high crystallinity and thickness control. Here, we present cm2 scale synthesis of 2H MoTe2 thin films with layer control and large grains that span several microns. Layer control is achieved by controlling the initial thickness of the precursor MoOx thin films, which are deposited on sapphire substrates by atomic layer deposition and subsequently tellurized. Despite the van der Waals epitaxy, the precursor-substrate interface is found to critically determine the uniformity in thickness and grain size of the resulting MoTe2 films: MoTe2 grown on sapphire show uniform films while MoTe2 grown on amorphous SiO2 substrates form islands. This synthesis strategy decouples the layer control from the variabilities of growth conditions for robust growth results, and is applicable to grow other transition metal dichalcogenides with layer control.

preprint2020arXiv

Comparison and Benchmarking of AI Models and Frameworks on Mobile Devices

Due to increasing amounts of data and compute resources, deep learning achieves many successes in various domains. The application of deep learning on the mobile and embedded devices is taken more and more attentions, benchmarking and ranking the AI abilities of mobile and embedded devices becomes an urgent problem to be solved. Considering the model diversity and framework diversity, we propose a benchmark suite, AIoTBench, which focuses on the evaluation of the inference abilities of mobile and embedded devices. AIoTBench covers three typical heavy-weight networks: ResNet50, InceptionV3, DenseNet121, as well as three light-weight networks: SqueezeNet, MobileNetV2, MnasNet. Each network is implemented by three frameworks which are designed for mobile and embedded devices: Tensorflow Lite, Caffe2, Pytorch Mobile. To compare and rank the AI capabilities of the devices, we propose two unified metrics as the AI scores: Valid Images Per Second (VIPS) and Valid FLOPs Per Second (VOPS). Currently, we have compared and ranked 5 mobile devices using our benchmark. This list will be extended and updated soon after.

preprint2020arXiv

Computation and data driven discovery of topological phononic materials

The discovery of topological quantum states marks a new chapter in both condensed matter physics and materials sciences. By analogy to spin electronic system, topological concepts have been extended into phonons, boosting the birth of topological phononics (TPs). Here, we present a high-throughput screening and data-driven approach to compute and evaluate TPs among over 10,000 materials. We have clarified 5014 TP materials and classified them into single Weyl, high degenerate Weyl, and nodal-line (ring) TPs. Among them, three representative cases of TPs have been discussed in detail. Furthermore, we suggest 322 TP materials with potential clean nontrivial surface states, which are favorable for experimental characterizations. This work significantly increases the current library of TP materials, which enables an in-depth investigation of their structure-property relations and opens new avenues for future device design related to TPs.

preprint2020arXiv

Dark matter, electroweak phase transition and gravitational wave in the type-II two-Higgs-doublet model with a singlet scalar field

In the framework of type-II two-Higgs-doublet model with a singlet scalar dark matter $S$, we study the dark matter observables, the electroweak phase transition, and the gravitational wave signals by such strongly first order phase transition after imposing the constraints of the LHC Higgs data. We take the heavy CP-even Higgs $H$ as the only portal between the dark matter and SM sectors, and find the LHC Higgs data and dark matter observables require $m_S$ and $m_H$ to be larger than 130 GeV and 360 GeV for $m_A=600$ GeV in the case of the 125 GeV Higgs with the SM-like coupling. Next, we carve out some parameter space where a strongly first order electroweak phase transition can be achieved, and find benchmark points for which the amplitudes of gravitational wave spectra reach the sensitivities of the future gravitational wave detectors.

preprint2020arXiv

Deep Learning based HEp-2 Image Classification: A Comprehensive Review

Classification of HEp-2 cell patterns plays a significant role in the indirect immunofluorescence test for identifying autoimmune diseases in the human body. Many automatic HEp-2 cell classification methods have been proposed in recent years, amongst which deep learning based methods have shown impressive performance. This paper provides a comprehensive review of the existing deep learning based HEp-2 cell image classification methods. These methods perform HEp-2 image classification at two levels, namely, cell-level and specimen-level. Both levels are covered in this review. At each level, the methods are organized with a deep network usage based taxonomy. The core idea, notable achievements, and key strengths and weaknesses of each method are critically analyzed. Furthermore, a concise review of the existing HEp-2 datasets that are commonly used in the literature is given. The paper ends with a discussion on novel opportunities and future research directions in this field. It is hoped that this paper would provide readers with a thorough reference of this novel, challenging, and thriving field.

preprint2020arXiv

Enhanced Solar Water Splitting by Swift Charge Separation in Au/FeOOH Sandwiched Single Crystalline Fe$_2$O$_3$ Nanoflake Photoelectrodes

In this work, single crystalline $α$-Fe$_2$O$_3$ nanoflakes (NFs) are formed in a highly dense array by Au seeding of a Fe substrate by a thermal oxidation technique. The NFs are conformally decorated with a thin FeOOH cocatalyst layer. Photoelectrochemical (PEC) measurements show that this photoanode with the $α$-Fe$_2$O$_3$/FeOOH NFs rooted on the Au/Fe structure exhibits a significantly enhanced PEC water oxidation performance compared to the plain $α$-Fe$_2$O$_3$ nanostructure on the Fe substrate. The $α$-Fe$_2$O$_3$/FeOOH NFs on Au/Fe photoanode yields a photocurrent density of 3.1 mA cm-2 at 1.5 VRHE, and a remarkably low onset potential of 0.5-0.6 VRHE in 1 M KOH under AM 1.5G (100 mW cm-2) simulated sunlight illumination. The enhancement in PEC performance can be attributed to a synergistic effect of the FeOOH top decoration and Au under-layer. While FeOOH facilitates hole transfer at the interface of electrode/electrolyte, the Au layer provides a sink for the electron transport to the back contact: this leads overall to a drastically improved charge-separation efficiency in the single crystalline $α$-Fe$_2$O$_3$ NF photoanode.

preprint2020arXiv

Enhancing Rumor Detection in Social Media Using Dynamic Propagation Structures

Social media, such as Facebook and Twitter, has become one of the most important channels for information dissemination. However, these social media platforms are often misused to spread rumors, which has brought about severe social problems, and consequently, there are urgent needs for automatic rumor detection techniques. Existing work on rumor detection concentrates more on the utilization of textual features, but diffusion structure itself can provide critical propagating information in identifying rumors. Previous works which have considered structural information, only utilize limited propagation structures. Moreover, few related research has considered the dynamic evolution of diffusion structures. To address these issues, in this paper, we propose a Neural Model using Dynamic Propagation Structures (NM-DPS) for rumor detection in social media. Firstly, we propose a partition approach to model the dynamic evolution of propagation structure and then use temporal attention based neural model to learn a representation for the dynamic structure. Finally, we fuse the structure representation and content features into a unified framework for effective rumor detection. Experimental results on two real-world social media datasets demonstrate the salience of dynamic propagation structure information and the effectiveness of our proposed method in capturing the dynamic structure.

preprint2020arXiv

Exploration of Input Patterns for Enhancing the Performance of Liquid State Machines

Spiking Neural Networks (SNN) have gained increasing attention for its low power consumption. But training SNN is challenging. Liquid State Machine (LSM), as a major type of Reservoir computing, has been widely recognized for its low training cost among SNNs. The exploration of LSM topology for enhancing performance often requires hyper-parameter search, which is both resource-expensive and time-consuming. We explore the influence of input scale reduction on LSM instead. There are two main reasons for studying input reduction of LSM. One is that the input dimension of large images requires efficient processing. Another one is that input exploration is generally more economic than architecture search. To mitigate the difficulty in effectively dealing with huge input spaces of LSM, and to find that whether input reduction can enhance LSM performance, we explore several input patterns, namely fullscale, scanline, chessboard, and patch. Several datasets have been used to evaluate the performance of the proposed input patterns, including two spatio image datasets and one spatio-temporal image database. The experimental results show that the reduced input under chessboard pattern improves the accuracy by up to 5%, and reduces execution time by up to 50% with up to 75\% less input storage than the fullscale input pattern for LSM.

preprint2020arXiv

Exploration of Surgeons' Natural Skills for Robotic Catheterization

Despite having the robotic catheter systems which have recently emerged as safe way of performing cardiovascular interventions, a number of important challenges are yet to be investigated. One of them is exploration of surgeons' natural skills during vascular catheterization with robotic systems. In this study, surgeons' natural hand motions were investigated for identification of four basic movements used for intravascular catheterization. Controlled experiment was setup to acquire surface electromyography (sEMG) signals from six muscles that are innervated when a subject with catheterization skills made the four movements in open settings. k-means and k-NN models were implemented over average EMG and root means square features to uniquely identify the movements. The result shows great potentials of sEMG analysis towards designing intelligent cyborg control for safe and efficient robotic catheterization.

preprint2020arXiv

Extended Batch Normalization

Batch normalization (BN) has become a standard technique for training the modern deep networks. However, its effectiveness diminishes when the batch size becomes smaller, since the batch statistics estimation becomes inaccurate. That hinders batch normalization's usage for 1) training larger model which requires small batches constrained by memory consumption, 2) training on mobile or embedded devices of which the memory resource is limited. In this paper, we propose a simple but effective method, called extended batch normalization (EBN). For NCHW format feature maps, extended batch normalization computes the mean along the (N, H, W) dimensions, as the same as batch normalization, to maintain the advantage of batch normalization. To alleviate the problem caused by small batch size, extended batch normalization computes the standard deviation along the (N, C, H, W) dimensions, thus enlarges the number of samples from which the standard deviation is computed. We compare extended batch normalization with batch normalization and group normalization on the datasets of MNIST, CIFAR-10/100, STL-10, and ImageNet, respectively. The experiments show that extended batch normalization alleviates the problem of batch normalization with small batch size while achieving close performances to batch normalization with large batch size.

preprint2020arXiv

Finet: Using Fine-grained Batch Normalization to Train Light-weight Neural Networks

To build light-weight network, we propose a new normalization, Fine-grained Batch Normalization (FBN). Different from Batch Normalization (BN), which normalizes the final summation of the weighted inputs, FBN normalizes the intermediate state of the summation. We propose a novel light-weight network based on FBN, called Finet. At training time, the convolutional layer with FBN can be seen as an inverted bottleneck mechanism. FBN can be fused into convolution at inference time. After fusion, Finet uses the standard convolution with equal channel width, thus makes the inference more efficient. On ImageNet classification dataset, Finet achieves the state-of-art performance (65.706% accuracy with 43M FLOPs, and 73.786% accuracy with 303M FLOPs), Moreover, experiments show that Finet is more efficient than other state-of-art light-weight networks.

preprint2020arXiv

Globalized distributionally robust optimization problems under the moment-based framework

This paper is devoted to reduce the conservatism of distributionally robust optimization with moments information. Since the optimal solution of distributionally robust optimization is required to be feasible for all uncertain distributions in a given ambiguity distribution set and so the conservatism of the optimal solution is inevitable. To address this issue, we introduce the globalized distributionally robust counterpart (GDRC) which allows constraint violations controlled by functional distance of the true distribution to the inner uncertainty distribution set. We obtain the deterministic equivalent forms for several GDRCs under the moment-based framework. To be specific, we show the deterministic equivalent system of inequalities for the GDRCs under second order moment information with a separable distance function and a jointly convex distance function, respectively. Moreover, the feasible set of the system is convex. We also develop the deterministic equivalent inequality for the GDRC under first order moment and support information. The computationally tractable examples are presented for these GDRCs. A numerical tests of a portfolio optimization problem is given to show the efficiency of our methods and the results demonstrate that the globalized distributionally robust solutions is non-conservative and flexible compared to the distributionally robust solutions.

preprint2020arXiv

GreyReID: A Two-stream Deep Framework with RGB-grey Information for Person Re-identification

In this paper, we observe that most false positive images (i.e., different identities with query images) in the top ranking list usually have the similar color information with the query image in person re-identification (Re-ID). Meanwhile, when we use the greyscale images generated from RGB images to conduct the person Re-ID task, some hard query images can obtain better performance compared with using RGB images. Therefore, RGB and greyscale images seem to be complementary to each other for person Re-ID. In this paper, we aim to utilize both RGB and greyscale images to improve the person Re-ID performance. To this end, we propose a novel two-stream deep neural network with RGB-grey information, which can effectively fuse RGB and greyscale feature representations to enhance the generalization ability of Re-ID. Firstly, we convert RGB images to greyscale images in each training batch. Based on these RGB and greyscale images, we train the RGB and greyscale branches, respectively. Secondly, to build up connections between RGB and greyscale branches, we merge the RGB and greyscale branches into a new joint branch. Finally, we concatenate the features of all three branches as the final feature representation for Re-ID. Moreover, in the training process, we adopt the joint learning scheme to simultaneously train each branch by the independent loss function, which can enhance the generalization ability of each branch. Besides, a global loss function is utilized to further fine-tune the final concatenated feature. The extensive experiments on multiple benchmark datasets fully show that the proposed method can outperform the state-of-the-art person Re-ID methods. Furthermore, using greyscale images can indeed improve the person Re-ID performance.

preprint2020arXiv

HPC AI500: The Methodology, Tools, Roofline Performance Models, and Metrics for Benchmarking HPC AI Systems

The recent years witness a trend of applying large-scale distributed deep learning in both business and scientific computing areas, whose goal is to speed up the training time to achieve a state-of-the-art quality. The HPC community feels a great interest in building the HPC AI systems that are dedicated to running those workloads. The HPC AI benchmarks accelerate the process. Unfortunately, benchmarking HPC AI systems at scale raises serious challenges. None of previous HPC AI benchmarks achieve the goal of being equivalent, relevant, representative, affordable, and repeatable. This paper presents a comprehensive methodology, tools, Roofline performance models, and innovative metrics for benchmarking, optimizing, and ranking HPC AI systems, which we call HPC AI500 V2.0. We abstract the HPC AI system into nine independent layers, and present explicit benchmarking rules and procedures to assure equivalence of each layer, repeatability, and replicability. On the basis of AIBench -- by far the most comprehensive AI benchmarks suite, we present and build two HPC AI benchmarks from both business and scientific computing: Image Classification, and Extreme Weather Analytics, achieving both representativeness and affordability. To rank the performance and energy-efficiency of HPC AI systems, we propose Valid FLOPS, and Valid FLOPS per watt, which impose a penalty on failing to achieve the target quality. We propose using convolution and GEMM -- the two most intensively-used kernel functions to measure the upper bound performance of the HPC AI systems, and present HPC AI roofline models for guiding performance optimizations. The evaluations show our methodology, benchmarks, performance models, and metrics can measure, optimize, and rank the HPC AI systems in a scalable, simple, and affordable way. HPC AI500 V2.0 are publicly available from http://www.benchcouncil.org/benchhub/hpc-ai500-benchmark.

preprint2020arXiv

Human Activity Recognition based on Dynamic Spatio-Temporal Relations

Human activity, which usually consists of several actions, generally covers interactions among persons and or objects. In particular, human actions involve certain spatial and temporal relationships, are the components of more complicated activity, and evolve dynamically over time. Therefore, the description of a single human action and the modeling of the evolution of successive human actions are two major issues in human activity recognition. In this paper, we develop a method for human activity recognition that tackles these two issues. In the proposed method, an activity is divided into several successive actions represented by spatio temporal patterns, and the evolution of these actions are captured by a sequential model. A refined comprehensive spatio temporal graph is utilized to represent a single action, which is a qualitative representation of a human action incorporating both the spatial and temporal relations of the participant objects. Next, a discrete hidden Markov model is applied to model the evolution of action sequences. Moreover, a fully automatic partition method is proposed to divide a long-term human activity video into several human actions based on variational objects and qualitative spatial relations. Finally, a hierarchical decomposition of the human body is introduced to obtain a discriminative representation for a single action. Experimental results on the Cornell Activity Dataset demonstrate the efficiency and effectiveness of the proposed approach, which will enable long videos of human activity to be better recognized.

preprint2020arXiv

Industrial Scale Privacy Preserving Deep Neural Network

Deep Neural Network (DNN) has been showing great potential in kinds of real-world applications such as fraud detection and distress prediction. Meanwhile, data isolation has become a serious problem currently, i.e., different parties cannot share data with each other. To solve this issue, most research leverages cryptographic techniques to train secure DNN models for multi-parties without compromising their private data. Although such methods have strong security guarantee, they are difficult to scale to deep networks and large datasets due to its high communication and computation complexities. To solve the scalability of the existing secure Deep Neural Network (DNN) in data isolation scenarios, in this paper, we propose an industrial scale privacy preserving neural network learning paradigm, which is secure against semi-honest adversaries. Our main idea is to split the computation graph of DNN into two parts, i.e., the computations related to private data are performed by each party using cryptographic techniques, and the rest computations are done by a neutral server with high computation ability. We also present a defender mechanism for further privacy protection. We conduct experiments on real-world fraud detection dataset and financial distress prediction dataset, the encouraging results demonstrate the practicalness of our proposal.

preprint2020arXiv

Initial-Value Privacy of Linear Dynamical Systems

This paper studies initial-value privacy problems of linear dynamical systems. We consider a standard linear time-invariant system with random process and measurement noises. For such a system, eavesdroppers having access to system output trajectories may infer the system initial states, leading to initial-value privacy risks. When a finite number of output trajectories are eavesdropped, we consider a requirement that any guess about the initial values can be plausibly denied. When an infinite number of output trajectories are eavesdropped, we consider a requirement that the initial values should not be uniquely recoverable. In view of these two privacy requirements, we define differential initial-value privacy and intrinsic initial-value privacy, respectively, for the system as metrics of privacy risks. First of all, we prove that the intrinsic initial-value privacy is equivalent to unobservability, while the differential initial-value privacy can be achieved for a privacy budget depending on an extended observability matrix of the system and the covariance of the noises. Next, the inherent network nature of the considered linear system is explored, where each individual state corresponds to a node and the state and output matrices induce interaction and sensing graphs, leading to a network system. Under this network system perspective, we allow the initial states at some nodes to be public, and investigate the resulting intrinsic initial-value privacy of each individual node. We establish necessary and sufficient conditions for such individual node initial-value privacy, and also prove that the intrinsic initial-value privacy of individual nodes is generically determined by the network structure. These results may be extended to linear systems with time-varying dynamics under the same analysis framework.

preprint2020arXiv

Method for Extracting Patterns of Coordinated Network Attacks on Electric Power CPS based on Temporal-Topological Correlation

In the analysis of coordinated network attacks on electric power cyber-physical system (CPS), it is difficult to restore the complete attack path, and the intent of the attack cannot be identified automatically. A method is therefore proposed for the extracting patterns of coordinated network attacks on electric power CPS based on temporal-topological correlation. First, the attack events are aggregated according to the alarm log of the cyber space, and a temporal-causal Bayesian network-based cyber attack recognition algorithm is proposed to parse out the cyber attack sequences of the same attacker. Then, according to the characteristic curves of different attack measurement data in physical space, a combination of physical attack event criteria algorithm is designed to distinguish the types of physical attack events. Finally, physical attack events and cyber attack sequences are matched via temporal-topological correlation, frequent patterns of attack sequences are extracted, and hidden multi-step attack patterns are found from scattered grid measurement data and information from alarm logs. The effectiveness and efficiency of the proposed method are verified by the testbed at Mississippi State University.

preprint2020arXiv

MODEL: Motif-based Deep Feature Learning for Link Prediction

Link prediction plays an important role in network analysis and applications. Recently, approaches for link prediction have evolved from traditional similarity-based algorithms into embedding-based algorithms. However, most existing approaches fail to exploit the fact that real-world networks are different from random networks. In particular, real-world networks are known to contain motifs, natural network building blocks reflecting the underlying network-generating processes. In this paper, we propose a novel embedding algorithm that incorporates network motifs to capture higher-order structures in the network. To evaluate its effectiveness for link prediction, experiments were conducted on three types of networks: social networks, biological networks, and academic networks. The results demonstrate that our algorithm outperforms both the traditional similarity-based algorithms by 20% and the state-of-the-art embedding-based algorithms by 19%.

preprint2020arXiv

Moiré metrology of energy landscapes in van der Waals heterostructures

The emerging field of twistronics, which harnesses the twist angle between two-dimensional materials, represents a promising route for the design of quantum materials, as the twist-angle-induced superlattices offer means to control topology and strong correlations. At the small twist limit, and particularly under strain, as atomic relaxation prevails, the emergent moiré superlattice encodes elusive insights into the local interlayer interaction. Here we introduce moiré metrology as a combined experiment-theory framework to probe the stacking energy landscape of bilayer structures at the 0.1 meV/atom scale, outperforming the gold-standard of quantum chemistry. Through studying the shapes of moiré domains with numerous nano-imaging techniques, and correlating with multi-scale modelling, we assess and refine first-principle models for the interlayer interaction. We document the prowess of moiré metrology for three representative twisted systems: bilayer graphene, double bilayer graphene and H-stacked $MoSe_2/WSe_2$. Moiré metrology establishes sought after experimental benchmarks for interlayer interaction, thus enabling accurate modelling of twisted multilayers.

preprint2020arXiv

Neural Canonical Transformation with Symplectic Flows

Canonical transformation plays a fundamental role in simplifying and solving classical Hamiltonian systems. We construct flexible and powerful canonical transformations as generative models using symplectic neural networks. The model transforms physical variables towards a latent representation with an independent harmonic oscillator Hamiltonian. Correspondingly, the phase space density of the physical system flows towards a factorized Gaussian distribution in the latent space. Since the canonical transformation preserves the Hamiltonian evolution, the model captures nonlinear collective modes in the learned latent representation. We present an efficient implementation of symplectic neural coordinate transformations and two ways to train the model. The variational free energy calculation is based on the analytical form of physical Hamiltonian. While the phase space density estimation only requires samples in the coordinate space for separable Hamiltonians. We demonstrate appealing features of neural canonical transformation using toy problems including two-dimensional ring potential and harmonic chain. Finally, we apply the approach to real-world problems such as identifying slow collective modes in alanine dipeptide and conceptual compression of the MNIST dataset.

preprint2020arXiv

Nonlinear imaging with all-dielectric metasurfaces

Nonlinear metasurfaces incorporate many of the functionalities of their linear counterparts such as wavefront shaping but simultaneously they perform nonlinear optical transformations. This dual functionality leads to a rather unintuitive physical behavior which is still widely unexplored for many photonic applications. The nonlinear processes render some basic principles governing the functionality of linear metasurfaces not directly applicable, such as the superposition principle and the geometric optics approximation. On the other hand, nonlinear metasurfaces facilitate new phenomena that are not possible in the linear regime. Here, we study the imaging of objects through a dielectric nonlinear metalens. We illuminate objects by infrared light and record their generated images at the visible third-harmonic wavelengths. We revisit the classical lens theory and suggest a generalized Gaussian lens equation for nonlinear imaging, verified both experimentally and analytically. We also demonstrate experimentally higher-order spatial correlations facilitated by the nonlinear metalens, resulting in additional image features.

preprint2020arXiv

Photoanodes Based on TiO$_2$ and $α$-Fe$_2$O$_3$ for Solar Water Splitting Superior Role of 1D Nanoarchitectures and of Combined Heterostructures

Solar driven photoelectrochemical water splitting (PEC-WS) using semiconductor photoelectrodes represents a promising approach for a sustainable and environmentally friendly production of renewable energy vectors and fuel sources, such as dihydrogen (H2). In this context, titanium dioxide (TiO$_2$) and iron oxide (hematite, $α$-Fe$_2$O$_3$) are among the most investigated candidates as photoanode materials, mainly owing to their resistance to photocorrosion, non-toxicity, natural abundance, and low production cost. Major drawbacks are, however, an inherently low electrical conductivity and a limited hole diffusion length that significantly affect the performance of TiO$_2$ and $α$-Fe$_2$O$_3$ in PEC devices. To this regard, one-dimensional (1D) nanostructuring is typically applied as it provides several superior features such as a significant enlargement of the material surface area, extended contact between the semiconductor and the electrolyte and, most remarkably, preferential electrical transport that overall suppress charge carrier recombination and improve TiO$_2$ and $α$-Fe$_2$O$_3$ photo-electrocatalytic properties. The present review describes various synthetic methods, properties and PEC applications of 1D-photoanodes (nanotubes, nanorods, nanofibers, nanowires) based on titania, hematite, and on $α$-Fe$_2$O$_3$/TiO$_2$ heterostructures. Various routes towards modification and enhancement of PEC activity of 1D photoanodes are also discussed including doping, decoration with co-catalysts and heterojunction engineering. Finally, the challenges related to the optimization of charge transfer kinetics in both oxides are highlighted.

preprint2020arXiv

Progressive Cross-camera Soft-label Learning for Semi-supervised Person Re-identification

In this paper, we focus on the semi-supervised person re-identification (Re-ID) case, which only has the intra-camera (within-camera) labels but not inter-camera (cross-camera) labels. In real-world applications, these intra-camera labels can be readily captured by tracking algorithms or few manual annotations, when compared with cross-camera labels. In this case, it is very difficult to explore the relationships between cross-camera persons in the training stage due to the lack of cross-camera label information. To deal with this issue, we propose a novel Progressive Cross-camera Soft-label Learning (PCSL) framework for the semi-supervised person Re-ID task, which can generate cross-camera soft-labels and utilize them to optimize the network. Concretely, we calculate an affinity matrix based on person-level features and adapt them to produce the similarities between cross-camera persons (i.e., cross-camera soft-labels). To exploit these soft-labels to train the network, we investigate the weighted cross-entropy loss and the weighted triplet loss from the classification and discrimination perspectives, respectively. Particularly, the proposed framework alternately generates progressive cross-camera soft-labels and gradually improves feature representations in the whole learning course. Extensive experiments on five large-scale benchmark datasets show that PCSL significantly outperforms the state-of-the-art unsupervised methods that employ labeled source domains or the images generated by the GAN-based models. Furthermore, the proposed method even has a competitive performance with respect to deep supervised Re-ID methods.

preprint2020arXiv

Residual-CycleGAN based Camera Adaptation for Robust Diabetic Retinopathy Screening

There are extensive researches focusing on automated diabetic reti-nopathy (DR) detection from fundus images. However, the accuracy drop is ob-served when applying these models in real-world DR screening, where the fun-dus camera brands are different from the ones used to capture the training im-ages. How can we train a classification model on labeled fundus images ac-quired from only one camera brand, yet still achieves good performance on im-ages taken by other brands of cameras? In this paper, we quantitatively verify the impact of fundus camera brands related domain shift on the performance of DR classification models, from an experimental perspective. Further, we pro-pose camera-oriented residual-CycleGAN to mitigate the camera brand differ-ence by domain adaptation and achieve increased classification performance on target camera images. Extensive ablation experiments on both the EyePACS da-taset and a private dataset show that the camera brand difference can signifi-cantly impact the classification performance and prove that our proposed meth-od can effectively improve the model performance on the target domain. We have inferred and labeled the camera brand for each image in the EyePACS da-taset and will publicize the camera brand labels for further research on domain adaptation.

preprint2020arXiv

SABRE and the Stawell Underground Physics Laboratory: Dark Matter Research at the Australian National University

The direct detection of dark matter is a key problem in astroparticle physics that generally requires the use of deep-underground laboratories for a low-background environment where the rare signals from dark matter interactions can be observed. This work reports on the Stawell Underground Physics Laboratory - currently under construction and the first such laboratory in the Southern Hemisphere - and the associated research program. A particular focus will be given to ANU's contribution to SABRE, a NaI:Tl dark matter direct detection experiment that aims to confirm or refute the long-standing DAMA result. Preliminary measurements of the NaI:Tl quenching factor and characterisation of the SABRE liquid scintillator veto are reported.

preprint2020arXiv

SDFN: Segmentation-based Deep Fusion Network for Thoracic Disease Classification in Chest X-ray Images

This study aims to automatically diagnose thoracic diseases depicted on the chest x-ray (CXR) images using deep convolutional neural networks. The existing methods generally used the entire CXR images for training purposes, but this strategy may suffer from two drawbacks. First, potential misalignment or the existence of irrelevant objects in the entire CXR images may cause unnecessary noise and thus limit the network performance. Second, the relatively low image resolution caused by the resizing operation, which is a common preprocessing procedure for training neural networks, may lead to the loss of image details, making it difficult to detect pathologies with small lesion regions. To address these issues, we present a novel method termed as segmentation-based deep fusion network (SDFN), which leverages the domain knowledge and the higherresolution information of local lung regions. Specifically, the local lung regions were identified and cropped by the Lung Region Generator (LRG). Two CNN-based classification models were then used as feature extractors to obtain the discriminative features of the entire CXR images and the cropped lung region images. Lastly, the obtained features were fused by the feature fusion module for disease classification. Evaluated by the NIH benchmark split on the Chest X-ray 14 Dataset, our experimental result demonstrated that the developed method achieved more accurate disease classification compared with the available approaches via the receiver operating characteristic (ROC) analyses. It was also found that the SDFN could localize the lesion regions more precisely as compared to the traditional method.

preprint2020arXiv

Secret Sharing based Secure Regressions with Applications

Nowadays, the utilization of the ever expanding amount of data has made a huge impact on web technologies while also causing various types of security concerns. On one hand, potential gains are highly anticipated if different organizations could somehow collaboratively share their data for technological improvements. On the other hand, data security concerns may arise for both data holders and data providers due to commercial or sociological concerns. To make a balance between technical improvements and security limitations, we implement secure and scalable protocols for multiple data holders to train linear regression and logistic regression models. We build our protocols based on the secret sharing scheme, which is scalable and efficient in applications. Moreover, our proposed paradigm can be generalized to any secure multiparty training scenarios where only matrix summation and matrix multiplications are used. We demonstrate our approach by experiments which shows the scalability and efficiency of our proposed protocols, and finally present its real-world applications.

preprint2020arXiv

SeqXFilter: A Memory-efficient Denoising Filter for Dynamic Vision Sensors

Neuromorphic event-based dynamic vision sensors (DVS) have much faster sampling rates and a higher dynamic range than frame-based imaging sensors. However, they are sensitive to background activity (BA) events that are unwanted. There are some filters for tackling this problem based on spatio-temporal correlation. However, they are either memory-intensive or computing-intensive. We propose \emph{SeqXFilter}, a spatio-temporal correlation filter with only a past event window that has an O(1) space complexity and has simple computations. We explore the spatial correlation of an event with its past few events by analyzing the distribution of the events when applying different functions on the spatial distances. We find the best function to check the spatio-temporal correlation for an event for \emph{SeqXFilter}, best separating real events and noise events. We not only give the visual denoising effect of the filter but also use two metrics for quantitatively analyzing the filter's performance. Four neuromorphic event-based datasets, recorded from four DVS with different output sizes, are used for validation of our method. The experimental results show that \emph{SeqXFilter} achieves similar performance as baseline NNb filters, but with extremely small memory cost and simple computation logic.

preprint2020arXiv

Shifu2: A Network Representation Learning Based Model for Advisor-advisee Relationship Mining

The advisor-advisee relationship represents direct knowledge heritage, and such relationship may not be readily available from academic libraries and search engines. This work aims to discover advisor-advisee relationships hidden behind scientific collaboration networks. For this purpose, we propose a novel model based on Network Representation Learning (NRL), namely Shifu2, which takes the collaboration network as input and the identified advisor-advisee relationship as output. In contrast to existing NRL models, Shifu2 considers not only the network structure but also the semantic information of nodes and edges. Shifu2 encodes nodes and edges into low-dimensional vectors respectively, both of which are then utilized to identify advisor-advisee relationships. Experimental results illustrate improved stability and effectiveness of the proposed model over state-of-the-art methods. In addition, we generate a large-scale academic genealogy dataset by taking advantage of Shifu2.

preprint2020arXiv

SNEAP: A Fast and Efficient Toolchain for Mapping Large-Scale Spiking Neural Network onto NoC-based Neuromorphic Platform

Spiking neural network (SNN), as the third generation of artificial neural networks, has been widely adopted in vision and audio tasks. Nowadays, many neuromorphic platforms support SNN simulation and adopt Network-on-Chips (NoC) architecture for multi-cores interconnection. However, interconnection brings huge area overhead to the platform. Moreover, run-time communication on the interconnection has a significant effect on the total power consumption and performance of the platform. In this paper, we propose a toolchain called SNEAP for mapping SNNs to neuromorphic platforms with multi-cores, which aims to reduce the energy and latency brought by spike communication on the interconnection. SNEAP includes two key steps: partitioning the SNN to reduce the spikes communicated between partitions, and mapping the partitions of SNN to the NoC to reduce average hop of spikes under the constraint of hardware resources. SNEAP can reduce more spikes communicated on the interconnection of NoC and spend less time than other toolchains in the partitioning phase. Moreover, the average hop of spikes is reduced more by SNEAP within a time period, which effectively reduces the energy and latency on the NoC-based neuromorphic platform. The experimental results show that SNEAP can achieve 418x reduction in end-to-end execution time, and reduce energy consumption and spike latency, on average, by 23% and 51% respectively, compared with SpiNeMap.

preprint2020arXiv

Structure-driven intercalated architecture of septuple-atomic-layer $MA_2Z_4$ family with diverse properties from semiconductor to topological insulator to Ising superconductor

Motivated by the fact that septuple-atomic-layer MnBi$_2$Te$_4$ can be structurally viewed as the combination of double-atomic-layer MnTe intercalating into quintuple-atomic-layer Bi$_2$Te$_3$, we present a general approach of constructing twelve septuple-atomic-layer $α_i$- and $β_i$-$MA_2Z_4$ monolayer family (\emph{i} = 1 to 6) by intercalating MoS$_2$-type $MZ$$_2$ monolayer into InSe-type A$_2$Z$_2$ monolayer. Besides reproducing the experimentally synthesized $α_1$-MoSi$_2$N$_4$, $α_1$-WSi$_2$N$_4$ and $β_5$-MnBi$_2$Te$_4$ monolayer materials, another 66 thermodynamically and dynamically stable $MA_2Z_4$ were predicted, which span a wide range of properties upon the number of valence electrons (VEC). $MA_2Z_4$ with the rules of 32 or 34 VEC are mostly semiconductors with direct or indirect band gap and, however, with 33 VEC are generally metal, half-metal ferromagnetism, or spin-gapless semiconductor upon whether or not an unpaired electron is spin polarized. Moreover, we propose $α_2$-WSi$_2$P$_4$ for the spin-valley polarization, $α_1$-TaSi$_2$N$_4$ for Ising superconductor and $β_2$-SrGa$_2$Se$_4$ for topological insulator.

preprint2020arXiv

Tensor network representations of parton wave functions

Tensor network states and parton wave functions are two pivotal methods for studying quantum many-body systems. This work connects these two subjects as we demonstrate that a variety of parton wave functions, such as projected Fermi sea and projected fermionic or bosonic paired states, can be represented exactly as tensor networks. The results can be compressed into matrix product states with moderate bond dimensions so various physical quantities can be computed efficiently. For the projected Fermi sea, we develop an excellent compression scheme with high fidelity using maximally localized Wannier orbitals. Numerical calculations on two parton wave functions demonstrate that our method exceeds commonly adopted Monte Carlo methods in some aspects. It produces energy and correlation function with very high accuracy that is difficult to achieve using Monte Carlo method. The entanglement measures that were almost impossible to compute before can also be obtained easily using our method.

preprint2020arXiv

The Collectivity of Heavy Mesons in Proton-Nucleus Collisions

Using a model based on the Color Glass Condensate framework and the dilute-dense factorization, we systematically study the azimuthal angular correlations between a heavy flavor meson and a light reference particle in proton-nucleus collisions. The obtained second harmonic coefficients (also known as the elliptic flows) for $J/ψ$ and $D^0$ agree with recent experimental data from the LHC. We also provide predictions for the elliptic flows of $Υ$ and $B$ meson, which can be measured in the near future at the LHC. This work can shed light on the physics origin of the collectivity phenomenon in the collisions of small systems.

preprint2020arXiv

Topological Thouless Pumping of Ultracold Fermions

A gas of electrons in a one-dimensional periodic potential can be transported even in the absence of a voltage bias if the potential is modulated slowly and periodically in time. Remarkably, the transferred charge per cycle is only sensitive to the topology of the path in parameter space. Although this so-called Thouless charge pump has first been proposed more than thirty years ago, it has not yet been realized. Here we report the first demonstration of topological Thouless pumping using ultracold atoms in a dynamically controlled optical superlattice. We observe a shift of the atomic cloud as a result of pumping and extract the topological invariance of the pumping process from this shift. We demonstrate the topological nature of the Thouless pump by varying the topology of the pumping path and verify that the topological pump indeed works in the quantum region by varying speed and temperature.

preprint2019arXiv

Machine Learning Holographic Mapping by Neural Network Renormalization Group

The exact holographic mapping (EHM) provides an explicit duality map between a conformal field theory (CFT) configuration and a massive field propagating on an emergent classical geometry. However, designing the optimal holographic mapping is challenging. Here we introduce the neural network renormalization group as a universal approach to design generic EHM for interacting field theories. Given a field theory action, we train a flow-based hierarchical deep generative neural network to reproduce the boundary field ensemble from uncorrelated bulk field fluctuations. In this way, the neural network develops the optimal renormalization group transformations. Using the machine-designed EHM to map the CFT back to a bulk effective action, we determine the bulk geodesic distance from the residual mutual information. We apply this approach to the complex $ϕ^4$ theory in two-dimensional Euclidian spacetime in its critical phase, and show that the emergent bulk geometry matches the three-dimensional hyperbolic geometry.

Lei Wang

Quick read

Decide how to stay connected

How to connect with this researcher

Open a focused conversation when the fit is right

See the researcher in context

Building this graph slice

112 published item(s)

Delta Score Matters! Spatial Adaptive Multi Guidance in Diffusion Models

VulTriage: Triple-Path Context Augmentation for LLM-Based Vulnerability Detection

Modern applications of machine learning in quantum sciences

An Inexact Preconditioned Zeroth-order Proximal Method for Composite Optimization

An Initial Investigation of Neural Replay Simulator for Over-the-Air Adversarial Perturbations to Automatic Speaker Verification

Broadband miniaturized spectrometers with a van der Waals tunnel diode

A GOA-Based Fault-Tolerant Trajectory Tracking Control for an Underwater Vehicle of Multi-Thruster System without Actuator Saturation

Effect of temperature-dependent thermophysical properties on turbulent forced convection under constant heat flux boundary condition

Smoothing Gradient Tracking for Decentralized Optimization over the Stiefel Manifold with Non-smooth Regularizers

A Communication-Efficient and Privacy-Aware Distributed Algorithm for Sparse PCA

A joint explanation of W-mass and muon g-2 in 2HDM

A Medical Semantic-Assisted Transformer for Radiographic Report Generation

A New High Energy Efficiency Scheme Based on Two-Dimension Resource Blocks in Wireless Communication Systems

A Novel Mix-normalization Method for Generalizable Multi-source Person Re-identification

A Variance-Reduced Stochastic Gradient Tracking Algorithm for Decentralized Optimization with Orthogonality Constraints

Ab-initio study of interacting fermions at finite temperature with neural canonical transformation

Active and Passive Hybrid Detection Method for Power CPS False Data Injection Attacks with Improved AKF and GRU-CNN

An efficient thermal lattice Boltzmann method for simulating three-dimensional liquid-vapor phase change

Attitude estimation from vector measurements: Necessary and sufficient conditions and convergent observer design

CenGCN: Centralized Convolutional Networks with Vertex Imbalance for Scale-Free Graphs

Contrastive Centroid Supervision Alleviates Domain Shift in Medical Image Classification

Deep Transfer Learning with Graph Neural Network for Sensor-Based Human Activity Recognition

Dissipation-enabled hydrodynamic conductivity in a tunable bandgap semiconductor

Factorizations of almost simple orthogonal groups of plus type

Fast and Arbitrary Beam Pattern Design for RIS-Assisted Terahertz Wireless Communication

Fusing Higher-order Features in Graph Neural Networks for Skeleton-based Action Recognition

Graph Neural Network with Curriculum Learning for Imbalanced Node Classification

Hardy-Sobolev inequalities with distance to the boundary weight functions

Indirect Adaptive Control of Nonlinearly Parameterized Nonlinear Dissipative Systems

Instance Image Retrieval by Learning Purely From Within the Dataset

Learning Class-Agnostic Pseudo Mask Generation for Box-Supervised Semantic Segmentation

LibFewShot: A Comprehensive Library for Few-shot Learning

Machine Learning assisted excess noise suppression for continuous-variable quantum key distribution

Machine Learning Based Multimodal Neuroimaging Genomics Dementia Score for Predicting Future Conversion to Alzheimer&#39;s Disease

MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving

OLxPBench: Real-time, Semantically Consistent, and Domain-specific are Essential in Benchmarking, Designing, and Implementing HTAP Systems

Private, Efficient, and Accurate: Protecting Models Trained by Multi-party Learning with Differential Privacy

Progressive Hard-case Mining across Pyramid Levels for Object Detection

Projective-truncation-approximation study of the one-dimensional $ϕ^4$ lattice model

Pursuing the Precision Study for Color Glass Condensate in Forward Hadron Productions

Revealing the CO2 emission reduction of ridesplitting and its determinants based on real-world data

Scalable and Sparsity-Aware Privacy-Preserving K-means Clustering with Application to Fraud Detection

Self-consistent Gradient-like Eigen Decomposition in Solving Schrödinger Equations

StyTr$^2$: Image Style Transfer with Transformers

Testing gravitational redshift based on microwave frequency links onboard China Space Station

The Shigesada-Kawasaki-Teramoto cross-diffusion system beyond detailed balance

Three-dimensional study of double droplets impact on a wettability-patterned surface

Topological EEG Nonlinear Dynamics Analysis for Emotion Recognition

Two-dimensional Obstructed Atomic Insulators with Fractional Corner Charge in MA$_2$Z$_4$ Family

Two-stream Hierarchical Similarity Reasoning for Image-text Matching

An efficient HTS electromagnetic model combining thin-strip, homogeneous and multi-scale methods by T-A formulation

Crosslink-Net: Double-branch Encoder Segmentation Network via Fusing Vertical and Horizontal Convolutions

Decentralized Optimization Over the Stiefel Manifold by an Approximate Augmented Lagrangian Function

Differentially Private Distributed Computation via Public-Private Communication Networks

Distributed Algorithms that Solve Boolean Equations with Local and Differential Privacies

Fast Evaporation Enabled Ultrathin Polymeric Coatings on Nanoporous Substrates for Highly Permeable Membranes

Giant Crystal Hall Effect in Collinear Antiferromagnetic $γ$-FeMn

HPC AI500: Representative, Repeatable and Simple HPC AI Benchmarking

Modeling Method for the Coupling Relations of Microgrid Cyber-Physical Systems Driven by Hybrid Spatiotemporal Events

Multiscale analysis of crystal defect formation in rapid solidification of pure aluminium and aluminium-copper alloys

Network Representation Learning: From Traditional Feature Learning to Deep Learning

Novel Two-Dimensional Layered MSi$_2$N$_4$ (M = Mo, W): New Promising Thermal Management Materials

Robust I&I Adaptive Tracking Control of Systems with Nonlinear Parameterization: An ISS Perspective

Robust Implementable Regulator Design of General Linear Systems

Robust Output Feedback Stabilization of MIMO Invertible Nonlinear Systems with Output-Dependent Multipliers (extended version)

Robust Output Feedback Stabilization of Multivariable Invertible Nonlinear Systems: A Feedback Linearization-Based Method

Simulation of an imaging system for internal contamination of lungs using MPA-MURA coded aperture collimator

Tropical Tensor Network for Ground States of Spin Glasses

Unified First-Principles Study of the Anomalous Hall Effect Based on Exact Muffin-Tin Orbitals

A Neural Architecture Search based Framework for Liquid State Machine Design

A Noise Filter for Dynamic Vision Sensors using Self-adjusting Threshold

ASMFS: Adaptive-Similarity-based Multi-modality Feature Selection for Classification of Alzheimer&#39;s Disease

Machine Learning Based Multimodal Neuroimaging Genomics Dementia Score for Predicting Future Conversion to Alzheimer's Disease

ASMFS: Adaptive-Similarity-based Multi-modality Feature Selection for Classification of Alzheimer's Disease

Exploration of Surgeons' Natural Skills for Robotic Catheterization