Researcher profile

Hong Liu

Hong Liu contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate

Trust snapshot

Quick read

Trust 21 - EmergingVerification L1Unclaimed author

61works

0followers

34topics

4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2026arXiv

MoPO: Incorporating Motion Prior for Occluded Human Mesh Recovery

Although recent studies have made remarkable progress in human mesh recovery, they still exhibit limited robustness to occlusions and often produce inaccurate poses and severe motion jitter due to the insufficient spatial features for occluded body parts. Inspired by the rapid advancements in human motion prediction, we discover that compared to occluded image features, pose sequence inherently contains reliable motion prior for estimating occluded body parts. In this paper, we incorporate Motion Prior for Occluded human mesh recovery, called MoPO. Our MoPO mainly consists of two components: 1) The motion de-occlusion module, where we propose a spatial-temporal occlusion detector to detect joint visibility, and then we propose a lightweight motion predictor to complete the occluded body parts by predicting the most plausible joint positions based on history poses. 2) The motion-aware fusion and refinement module, which fuses the completed joint sequence with image features to estimate human shape and initial human pose. Moreover, the completed joint sequence is further used to refine the final human pose through inverse kinematics, which provides the occlusion-free motion prior for regressing human poses. Extensive experiments demonstrate that MoPO achieves state-of-the-art performance on both occlusion-specific and standard benchmarks, significantly enhancing the accuracy and temporal consistency of occluded human mesh recovery. Our code and demo can be found in the supplementary material.

preprint2023arXiv

A Gated Cross-domain Collaborative Network for Underwater Object Detection

Underwater object detection (UOD) plays a significant role in aquaculture and marine environmental protection. Considering the challenges posed by low contrast and low-light conditions in underwater environments, several underwater image enhancement (UIE) methods have been proposed to improve the quality of underwater images. However, only using the enhanced images does not improve the performance of UOD, since it may unavoidably remove or alter critical patterns and details of underwater objects. In contrast, we believe that exploring the complementary information from the two domains is beneficial for UOD. The raw image preserves the natural characteristics of the scene and texture information of the objects, while the enhanced image improves the visibility of underwater objects. Based on this perspective, we propose a Gated Cross-domain Collaborative Network (GCC-Net) to address the challenges of poor visibility and low contrast in underwater environments, which comprises three dedicated components. Firstly, a real-time UIE method is employed to generate enhanced images, which can improve the visibility of objects in low-contrast areas. Secondly, a cross-domain feature interaction module is introduced to facilitate the interaction and mine complementary information between raw and enhanced image features. Thirdly, to prevent the contamination of unreliable generated results, a gated feature fusion module is proposed to adaptively control the fusion ratio of cross-domain information. Our method presents a new UOD paradigm from the perspective of cross-domain information interaction and fusion. Experimental results demonstrate that the proposed GCC-Net achieves state-of-the-art performance on four underwater datasets.

preprint2023arXiv

A systematic formulation of chiral anomalous magnetohydrodynamics

We present a new way of deriving effective theories of dynamical electromagnetic fields in general media. It can be used to give a systematic formulation of magnetohydrodynamics (MHD) with strong magnetic fields, including systems with chiral matter and Adler-Bell-Jackiw (ABJ) anomaly. We work in the regime in which velocity and temperature fluctuations can be neglected. The resulting chiral anomalous MHD incorporates and generalizes the chiral magnetic effect, the chiral separation effect, the chiral electric separation effect, as well as recently derived strong-field MHD, all in a single coherent framework. At linearized level, the theory predicts that the chiral magnetic wave does not survive dynamical electromagnetic fields. A different chiral wave, to which we refer as the chiral magnetic electric separation wave, emerges as a result of dynamical versions of the chiral electric separation effect and the chiral magnetic effect. We predict its wave velocity. We also introduce a simple, but solvable nonlinear model to explore the fate of the chiral instability.

preprint2023arXiv

Achieving Domain Generalization in Underwater Object Detection by Domain Mixup and Contrastive Learning

The performance of existing underwater object detection methods degrades seriously when facing domain shift caused by complicated underwater environments. Due to the limitation of the number of domains in the dataset, deep detectors easily memorize a few seen domains, which leads to low generalization ability. There are two common ideas to improve the domain generalization performance. First, it can be inferred that the detector trained on as many domains as possible is domain-invariant. Second, for the images with the same semantic content in different domains, their hidden features should be equivalent. This paper further excavates these two ideas and proposes a domain generalization framework (named DMC) that learns how to generalize across domains from Domain Mixup and Contrastive Learning. First, based on the formation of underwater images, an image in an underwater environment is the linear transformation of another underwater environment. Thus, a style transfer model, which outputs a linear transformation matrix instead of the whole image, is proposed to transform images from one source domain to another, enriching the domain diversity of the training data. Second, mixup operation interpolates different domains on the feature level, sampling new domains on the domain manifold. Third, contrastive loss is selectively applied to features from different domains to force the model to learn domain invariant features but retain the discriminative capacity. With our method, detectors will be robust to domain shift. Also, a domain generalization benchmark S-UODAC2020 for detection is set up to measure the performance of our method. Comprehensive experiments on S-UODAC2020 and two object recognition benchmarks (PACS and VLCS) demonstrate that the proposed method is able to learn domain-invariant representations, and outperforms other domain generalization methods.

preprint2023arXiv

Many Hamiltonian subsets in large graphs with given density

A set of vertices in a graph is a Hamiltonian subset if it induces a subgraph containing a Hamiltonian cycle. Kim, Liu, Sharifzadeh and Staden proved that among all graphs with minimum degree $d$, $K_{d+1}$ minimises the number of Hamiltonian subsets. We prove a near optimal lower bound that takes also the order and the structure of a graph into account. For many natural graph classes, it provides a much better bound than the extremal one ($\approx 2^{d+1}$). Among others, our bound implies that an $n$-vertex $C_4$-free graphs with minimum degree $d$ contains at least $n2^{d^{2-o(1)}}$ Hamiltonian subsets.

preprint2023arXiv

Polynomial Schur's theorem

We resolve the Ramsey problem for $\{x,y,z:x+y=p(z)\}$ for all polynomials $p$ over $\mathbb{Z}$. In particular, we characterise all polynomials that are $2$-Ramsey, that is, those $p(z)$ such that any $2$-colouring of $\mathbb{N}$ contains infinitely many monochromatic solutions for $x+y=p(z)$. For polynomials that are not $2$-Ramsey, we characterise all $2$-colourings of $\mathbb{N}$ that are not $2$-Ramsey, revealing that certain divisibility barrier is the only obstruction to $2$-Ramseyness for $x+y=p(z)$.

preprint2022arXiv

Advancing Semi-Supervised Task Oriented Dialog Systems by JSA Learning of Discrete Latent Variable Models

Developing semi-supervised task-oriented dialog (TOD) systems by leveraging unlabeled dialog data has attracted increasing interests. For semi-supervised learning of latent state TOD models, variational learning is often used, but suffers from the annoying high-variance of the gradients propagated through discrete latent variables and the drawback of indirectly optimizing the target log-likelihood. Recently, an alternative algorithm, called joint stochastic approximation (JSA), has emerged for learning discrete latent variable models with impressive performances. In this paper, we propose to apply JSA to semi-supervised learning of the latent state TOD models, which is referred to as JSA-TOD. To our knowledge, JSA-TOD represents the first work in developing JSA based semi-supervised learning of discrete latent variable conditional models for such long sequential generation problems like in TOD systems. Extensive experiments show that JSA-TOD significantly outperforms its variational learning counterpart. Remarkably, semi-supervised JSA-TOD using 20% labels performs close to the full-supervised baseline on MultiWOZ2.1.

preprint2022arXiv

AO2-DETR: Arbitrary-Oriented Object Detection Transformer

Arbitrary-oriented object detection (AOOD) is a challenging task to detect objects in the wild with arbitrary orientations and cluttered arrangements. Existing approaches are mainly based on anchor-based boxes or dense points, which rely on complicated hand-designed processing steps and inductive bias, such as anchor generation, transformation, and non-maximum suppression reasoning. Recently, the emerging transformer-based approaches view object detection as a direct set prediction problem that effectively removes the need for hand-designed components and inductive biases. In this paper, we propose an Arbitrary-Oriented Object DEtection TRansformer framework, termed AO2-DETR, which comprises three dedicated components. More precisely, an oriented proposal generation mechanism is proposed to explicitly generate oriented proposals, which provides better positional priors for pooling features to modulate the cross-attention in the transformer decoder. An adaptive oriented proposal refinement module is introduced to extract rotation-invariant region features and eliminate the misalignment between region features and objects. And a rotation-aware set matching loss is used to ensure the one-to-one matching process for direct set prediction without duplicate predictions. Our method considerably simplifies the overall pipeline and presents a new AOOD paradigm. Comprehensive experiments on several challenging datasets show that our method achieves superior performance on the AOOD task.

preprint2022arXiv

Breast Cancer Molecular Subtypes Prediction on Pathological Images with Discriminative Patch Selecting and Multi-Instance Learning

Molecular subtypes of breast cancer are important references to personalized clinical treatment. For cost and labor savings, only one of the patient's paraffin blocks is usually selected for subsequent immunohistochemistry (IHC) to obtain molecular subtypes. Inevitable sampling error is risky due to tumor heterogeneity and could result in a delay in treatment. Molecular subtype prediction from conventional H&E pathological whole slide images (WSI) using AI method is useful and critical to assist pathologists pre-screen proper paraffin block for IHC. It's a challenging task since only WSI level labels of molecular subtypes can be obtained from IHC. Gigapixel WSIs are divided into a huge number of patches to be computationally feasible for deep learning. While with coarse slide-level labels, patch-based methods may suffer from abundant noise patches, such as folds, overstained regions, or non-tumor tissues. A weakly supervised learning framework based on discriminative patch selecting and multi-instance learning was proposed for breast cancer molecular subtype prediction from H&E WSIs. Firstly, co-teaching strategy was adopted to learn molecular subtype representations and filter out noise patches. Then, a balanced sampling strategy was used to handle the imbalance in subtypes in the dataset. In addition, a noise patch filtering algorithm that used local outlier factor based on cluster centers was proposed to further select discriminative patches. Finally, a loss function integrating patch with slide constraint information was used to finetune MIL framework on obtained discriminative patches and further improve the performance of molecular subtyping. The experimental results confirmed the effectiveness of the proposed method and our models outperformed even senior pathologists, with potential to assist pathologists to pre-screen paraffin blocks for IHC in clinic.

preprint2022arXiv

Clique immersion in graphs without fixed bipartite graph

A graph $G$ contains $H$ as an \emph{immersion} if there is an injective mapping $ϕ: V(H)\rightarrow V(G)$ such that for each edge $uv\in E(H)$, there is a path $P_{uv}$ in $G$ joining vertices $ϕ(u)$ and $ϕ(v)$, and all the paths $P_{uv}$, $uv\in E(H)$, are pairwise edge-disjoint. An analogue of Hadwiger's conjecture for the clique immersions by Lescure and Meyniel, and independently by Abu-Khzam and Langston, states that every graph $G$ contains $K_{χ(G)}$ as an immersion. We prove that for any constant $\varepsilon>0$ and integers $s,t\ge2$, there exists $d_0=d_0(\varepsilon,s,t)$ such that every $K_{s,t}$-free graph $G$ with $d(G)\ge d_0$ contains a clique immersion of order $(1-\varepsilon)d(G)$. This implies that the above-mentioned conjecture is asymptotically true for graphs without a fixed complete bipartite graph.

preprint2022arXiv

Contrastive Learning from Spatio-Temporal Mixed Skeleton Sequences for Self-Supervised Skeleton-Based Action Recognition

Self-supervised skeleton-based action recognition with contrastive learning has attracted much attention. Recent literature shows that data augmentation and large sets of contrastive pairs are crucial in learning such representations. In this paper, we found that directly extending contrastive pairs based on normal augmentations brings limited returns in terms of performance, because the contribution of contrastive pairs from the normal data augmentation to the loss get smaller as training progresses. Therefore, we delve into hard contrastive pairs for contrastive learning. Motivated by the success of mixing augmentation strategy which improves the performance of many tasks by synthesizing novel samples, we propose SkeleMixCLR: a contrastive learning framework with a spatio-temporal skeleton mixing augmentation (SkeleMix) to complement current contrastive learning approaches by providing hard contrastive samples. First, SkeleMix utilizes the topological information of skeleton data to mix two skeleton sequences by randomly combing the cropped skeleton fragments (the trimmed view) with the remaining skeleton sequences (the truncated view). Second, a spatio-temporal mask pooling is applied to separate these two views at the feature level. Third, we extend contrastive pairs with these two views. SkeleMixCLR leverages the trimmed and truncated views to provide abundant hard contrastive pairs since they involve some context information from each other due to the graph convolution operations, which allows the model to learn better motion representations for action recognition. Extensive experiments on NTU-RGB+D, NTU120-RGB+D, and PKU-MMD datasets show that SkeleMixCLR achieves state-of-the-art performance. Codes are available at https://github.com/czhaneva/SkeleMixCLR.

preprint2022arXiv

Disjoint isomorphic balanced clique subdivisions

A thoroughly studied problem in Extremal Graph Theory is to find the best possible density condition in a host graph $G$ for guaranteeing the presence of a particular subgraph $H$ in $G$. One such classical result, due to Bollobás and Thomason, and independently Komlós and Szemerédi, states that average degree $O(k^2)$ guarantees the existence of a $K_k$-subdivision. We study two directions extending this result. On the one hand, Verstraëte conjectured that the quadratic bound $O(k^2)$ would guarantee already two vertex-disjoint isomorphic copies of a $K_k$-subdivision. On the other hand, Thomassen conjectured that for each $k \in \mathbb{N}$ there is some $d = d(k)$ such that every graph with average degree at least $d$ contains a balanced subdivision of $K_k$, that is, a copy of $K_k$ where the edges are replaced by paths of equal length. Recently, Liu and Montgomery confirmed Thomassen's conjecture, but the optimal bound on $d(k)$ remains open. In this paper, we show that the quadratic bound $O(k^2)$ suffices to force a balanced $K_k$-subdivision. This gives the optimal bound on $d(k)$ needed in Thomassen's conjecture and implies the existence of $O(1)$ many vertex-disjoint isomorphic $K_k$-subdivisions, confirming Verstraëte's conjecture in a strong sense.

preprint2022arXiv

Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation

Despite the great progress in 3D human pose estimation from videos, it is still an open problem to take full advantage of a redundant 2D pose sequence to learn representative representations for generating one 3D pose. To this end, we propose an improved Transformer-based architecture, called Strided Transformer, which simply and effectively lifts a long sequence of 2D joint locations to a single 3D pose. Specifically, a Vanilla Transformer Encoder (VTE) is adopted to model long-range dependencies of 2D pose sequences. To reduce the redundancy of the sequence, fully-connected layers in the feed-forward network of VTE are replaced with strided convolutions to progressively shrink the sequence length and aggregate information from local contexts. The modified VTE is termed as Strided Transformer Encoder (STE), which is built upon the outputs of VTE. STE not only effectively aggregates long-range information to a single-vector representation in a hierarchical global and local fashion, but also significantly reduces the computation cost. Furthermore, a full-to-single supervision scheme is designed at both full sequence and single target frame scales applied to the outputs of VTE and STE, respectively. This scheme imposes extra temporal smoothness constraints in conjunction with the single target frame supervision and hence helps produce smoother and more accurate 3D poses. The proposed Strided Transformer is evaluated on two challenging benchmark datasets, Human3.6M and HumanEva-I, and achieves state-of-the-art results with fewer parameters. Code and models are available at \url{https://github.com/Vegetebird/StridedTransformer-Pose3D}.

preprint2022arXiv

Exponential decay of intersection volume with applications on list-decodability and Gilbert-Varshamov type bound

We give some natural sufficient conditions for balls in a metric space to have small intersection. Roughly speaking, this happens when the metric space is (i) expanding and (ii) well-spread, and (iii) a certain random variable on the boundary of a ball has a small tail. As applications, we show that the volume of intersection of balls in Hamming, Johnson spaces and symmetric groups decay exponentially as their centers drift apart. To verify condition (iii), we prove some large deviation inequalities `on a slice' for functions with Lipschitz conditions. We then use these estimates on intersection volumes to $\bullet$ obtain a sharp lower bound on list-decodability of random $q$-ary codes, confirming a conjecture of Li and Wootters; and $\bullet$ improve the classical bound of Levenshtein from 1971 on constant weight codes by a factor linear in dimension, resolving a problem raised by Jiang and Vardy. Our probabilistic point of view also offers a unified framework to obtain improvements on other Gilbert--Varshamov type bounds, giving conceptually simple and calculation-free proofs for $q$-ary codes, permutation codes, and spherical codes. Another consequence is a counting result on the number of codes, showing ampleness of large codes.

preprint2022arXiv

Facial-Sketch Synthesis: A New Challenge

This paper aims to conduct a comprehensive study on facial-sketch synthesis (FSS). However, due to the high costs of obtaining hand-drawn sketch datasets, there lacks a complete benchmark for assessing the development of FSS algorithms over the last decade. We first introduce a high-quality dataset for FSS, named FS2K, which consists of 2,104 image-sketch pairs spanning three types of sketch styles, image backgrounds, lighting conditions, skin colors, and facial attributes. FS2K differs from previous FSS datasets in difficulty, diversity, and scalability and should thus facilitate the progress of FSS research. Second, we present the largest-scale FSS investigation by reviewing 89 classical methods, including 25 handcrafted feature-based facial-sketch synthesis approaches, 29 general translation methods, and 35 image-to-sketch approaches. Besides, we elaborate comprehensive experiments on the existing 19 cutting-edge models. Third, we present a simple baseline for FSS, named FSGAN. With only two straightforward components, i.e., facial-aware masking and style-vector expansion, FSGAN surpasses the performance of all previous state-of-the-art models on the proposed FS2K dataset by a large margin. Finally, we conclude with lessons learned over the past years and point out several unsolved challenges. Our code is available at https://github.com/DengPingFan/FSGAN.

preprint2022arXiv

High-order Photonic Cavity Modes Enabled 3D Structural Color

It remains a challenge to directly print three-dimensional arbitrary shapes that exhibit structural colors at the micrometer scale. Woodpile photonic crystals (WPCs) fabricated via two-photon lithography (TPL) are promising as building blocks to produce 3D geometries that generate structural colors due to their ability to exhibit either omnidirectional or anisotropic photonic stopbands. However, existing approaches have focused on achieving structural colors when illuminating WPCs from the top, which necessitates print resolutions beyond the limit of commercial TPL and/or post-processing techniques. Here, we devised a new strategy to support high-order photonic cavity modes upon side-illumination on WPCs that surprisingly generate large reflectance peaks in the visible spectrum. Based on that, we demonstrate one-step printing of 3D photonic structural colors without requiring post-processing or subwavelength features. Vivid colors with reflectance peaks exhibiting a full width at half maximum of ~25 nm, a maximum reflectance of 50%, gamut of ~85% of sRGB, and large viewing angles, were achieved. In addition, we also demonstrated voxel-level manipulation and control of colors in arbitrary-shaped 3D objects constituted with WPCs as unit cells, which has great potential for applications in dynamic color displays, colorimetric sensing, anti-counterfeiting, and light-matter interaction platforms.

preprint2022arXiv

How to build a pillar: a proof of Thomassen's conjecture

Carsten Thomassen in 1989 conjectured that if a graph has minimum degree more than the number of atoms in the universe ($δ(G)\ge 10^{10^{10}}$), then it contains a pillar, which is a graph that consists of two vertex-disjoint cycles of the same length, $s$ say, along with $s$ vertex-disjoint paths of the same length which connect matching vertices in order around the cycles. Despite the simplicity of the structure of pillars and various developments of powerful embedding methods for paths and cycles in the past three decades, this innocent looking conjecture has seen no progress to date. In this paper, we give a proof of this conjecture by building a pillar (algorithmically) in sublinear expanders.

preprint2022arXiv

Identity-Sensitive Knowledge Propagation for Cloth-Changing Person Re-identification

Cloth-changing person re-identification (CC-ReID), which aims to match person identities under clothing changes, is a new rising research topic in recent years. However, typical biometrics-based CC-ReID methods often require cumbersome pose or body part estimators to learn cloth-irrelevant features from human biometric traits, which comes with high computational costs. Besides, the performance is significantly limited due to the resolution degradation of surveillance images. To address the above limitations, we propose an effective Identity-Sensitive Knowledge Propagation framework (DeSKPro) for CC-ReID. Specifically, a Cloth-irrelevant Spatial Attention module is introduced to eliminate the distraction of clothing appearance by acquiring knowledge from the human parsing module. To mitigate the resolution degradation issue and mine identity-sensitive cues from human faces, we propose to restore the missing facial details using prior facial knowledge, which is then propagated to a smaller network. After training, the extra computations for human parsing or face restoration are no longer required. Extensive experiments show that our framework outperforms state-of-the-art methods by a large margin. Our code is available at https://github.com/KimbingNg/DeskPro.

preprint2022arXiv

Learning Deep Direct-Path Relative Transfer Function for Binaural Sound Source Localization

Direct-path relative transfer function (DP-RTF) refers to the ratio between the direct-path acoustic transfer functions of two microphone channels. Though DP-RTF fully encodes the sound spatial cues and serves as a reliable localization feature, it is often erroneously estimated in the presence of noise and reverberation. This paper proposes to learn DP-RTF with deep neural networks for robust binaural sound source localization. A DP-RTF learning network is designed to regress the binaural sensor signals to a real-valued representation of DP-RTF. It consists of a branched convolutional neural network module to separately extract the inter-channel magnitude and phase patterns, and a convolutional recurrent neural network module for joint feature learning. To better explore the speech spectra to aid the DP-RTF estimation, a monaural speech enhancement network is used to recover the direct-path spectrograms from the noisy ones. The enhanced spectrograms are stacked onto the noisy spectrograms to act as the input of the DP-RTF learning network. We train one unique DP-RTF learning network using many different binaural arrays to enable the generalization of DP-RTF learning across arrays. This way avoids time-consuming training data collection and network retraining for a new array, which is very useful in practical application. Experimental results on both simulated and real-world data show the effectiveness of the proposed method for direction of arrival (DOA) estimation in the noisy and reverberant environment, and a good generalization ability to unseen binaural arrays.

preprint2022arXiv

MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation

Estimating 3D human poses from monocular videos is a challenging task due to depth ambiguity and self-occlusion. Most existing works attempt to solve both issues by exploiting spatial and temporal relationships. However, those works ignore the fact that it is an inverse problem where multiple feasible solutions (i.e., hypotheses) exist. To relieve this limitation, we propose a Multi-Hypothesis Transformer (MHFormer) that learns spatio-temporal representations of multiple plausible pose hypotheses. In order to effectively model multi-hypothesis dependencies and build strong relationships across hypothesis features, the task is decomposed into three stages: (i) Generate multiple initial hypothesis representations; (ii) Model self-hypothesis communication, merge multiple hypotheses into a single converged representation and then partition it into several diverged hypotheses; (iii) Learn cross-hypothesis communication and aggregate the multi-hypothesis features to synthesize the final 3D pose. Through the above processes, the final representation is enhanced and the synthesized pose is much more accurate. Extensive experiments show that MHFormer achieves state-of-the-art results on two challenging datasets: Human3.6M and MPI-INF-3DHP. Without bells and whistles, its performance surpasses the previous best result by a large margin of 3% on Human3.6M. Code and models are available at \url{https://github.com/Vegetebird/MHFormer}.

preprint2022arXiv

Mission Apollo: Landing Optical Circuit Switching at Datacenter Scale

In this paper, we describe Apollo, to the best of our knowledge, the world's first large-scale production deployment of optical circuit switches (OCSes) for datacenter networking. We will first describe the infrastructure challenges and use cases that motivated optical switching inside datacenters. We then delve into the requirements of OCSes for datacenter applications: balancing cost, port count, switching time, and optical performance, which drive design choices and implementation details of our internally developed 3D MEMS-based OCS. To enable the Apollo optical switching layer, we employ circulators to realize bidirectional links through the OCS, effectively doubling the OCS radix. The OCS and circulator design choices were critical for meeting network bandwidth, scale, and cost targets. We review the critical co-design of WDM transceiver technology for these OCS plus circulator-based bidirectional links and their corresponding physical impairments, delivered over four generations/speeds of optical interconnect. Finally, we conclude with thoughts on future directions in hardware development and associated applications.

preprint2022arXiv

Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition

Graph convolutional networks have been widely used for skeleton-based action recognition due to their excellent modeling ability of non-Euclidean data. As the graph convolution is a local operation, it can only utilize the short-range joint dependencies and short-term trajectory but fails to directly model the distant joints relations and long-range temporal information that are vital to distinguishing various actions. To solve this problem, we present a multi-scale spatial graph convolution (MS-GC) module and a multi-scale temporal graph convolution (MT-GC) module to enrich the receptive field of the model in spatial and temporal dimensions. Concretely, the MS-GC and MT-GC modules decompose the corresponding local graph convolution into a set of sub-graph convolution, forming a hierarchical residual architecture. Without introducing additional parameters, the features will be processed with a series of sub-graph convolutions, and each node could complete multiple spatial and temporal aggregations with its neighborhoods. The final equivalent receptive field is accordingly enlarged, which is capable of capturing both short- and long-range dependencies in spatial and temporal domains. By coupling these two modules as a basic block, we further propose a multi-scale spatial temporal graph convolutional network (MST-GCN), which stacks multiple blocks to learn effective motion representations for action recognition. The proposed MST-GCN achieves remarkable performance on three challenging benchmark datasets, NTU RGB+D, NTU-120 RGB+D and Kinetics-Skeleton, for skeleton-based action recognition.

preprint2022arXiv

PetLock:A Genderless and Standard Interface for the Future On-orbit Construction

Modular design is the foundation of on orbit construction technology of large space facilities in the future.Standard interface is the key technology of modular design of the future space robotic systems and space facilities.This paper presents the designed and tested of PetLock,a standard and genderless interface which can transfer mechanical loads,power and data between the future modular space robotic manipulator and spacecraft.PetLock adopts a completely genderless design,including connection face,locking mechanism,data and power interface.The connection surface provides a large translation and rotation misalignment tolerance,due to its 120-degree symmetrical and 3D shape design.The locking mechanism features the three locking pins retraction structure design,which is simple and reliable.POGO pin connectors in the center of the interface provides the power and data transfer capabilities.Due to the advantages of high locking force,large tolerance,high reliability and low cost,PetLock has the very big application potential in future on orbit construction missions.

preprint2022arXiv

Prototype Design and Efficiency Analysis of a Novel Robot Drive Based on 3K-H-V Topology

Robot actuators directly affect the performance of robots, and robot drives directly affect the performance of robot actuators. With the development of robotics, robots have put higher requirements on robot drives, such as high stiffness, high accuracy, high loading, high efficiency, low backlash, compact size, and hollow structure. In order to meet the demand development of robot actuators, this research base proposes a new robot drive based on 3K-H-V topology using involute and cycloidal gear shapes, planetary cycloidal drive, from the perspective of drive topology and through the design idea of decoupling. In this study, the reduction ratio and the efficiency model of the 3K-H-V topology were analyzed, and a prototype planetary cycloidal actuator was designed. The feasibility of the drive is initially verified by experimentally concluding that the PCA has a hollow structure, compact size, and high torque density (69 kg/Nm).

preprint2022arXiv

Recent advances and clinical applications of deep learning in medical image analysis

Deep learning has received extensive research interest in developing new medical image processing algorithms, and deep learning based models have been remarkably successful in a variety of medical imaging tasks to support disease detection and diagnosis. Despite the success, the further improvement of deep learning models in medical image analysis is majorly bottlenecked by the lack of large-sized and well-annotated datasets. In the past five years, many studies have focused on addressing this challenge. In this paper, we reviewed and summarized these recent studies to provide a comprehensive overview of applying deep learning methods in various medical image analysis tasks. Especially, we emphasize the latest progress and contributions of state-of-the-art unsupervised and semi-supervised deep learning in medical image analysis, which are summarized based on different application scenarios, including classification, segmentation, detection, and image registration. We also discuss the major technical challenges and suggest the possible solutions in future research efforts.

preprint2022arXiv

Recent Advances in Tunable Metasurfaces: Materials, Design and Applications

Metasurfaces, a two-dimensional (2D) form of metamaterials constituted by planar meta-atoms, exhibit exotic abilities to freely tailor electromagnetic (EM) waves. Over the past decade, tunable metasurfaces have come to the frontier in the field of nanophotonics, with tremendous effort focused on developing and integrating various active materials into metasurfaces. As a result, tunable/reconfigurable metasurfaces with multi-functionalities triggered by various external stimuli have been successfully demonstrated, openings a new avenue to dynamically manipulate and control EM waves for photonic applications in demand. In this review, we first brief the progress of tunable metasurfaces development in the last decade and highlight representative works from the perspectives of active materials development, design methodologies and application-driven exploration. Then, we elaborate on the active tuning mechanisms and relevant active materials. Next, we discuss recent achievements in theory as well as machine learning (ML) assisted design methodologies to sustain the development of this field. After that, we summarize and describe typical application areas of the tunable metasurfaces. We conclude this review by analyzing existing challenges and presenting our perspectives on future directions and opportunities in this vibrant and fast-developing field.

preprint2022arXiv

Self-supervised Learning is More Robust to Dataset Imbalance

Self-supervised learning (SSL) is a scalable way to learn general visual representations since it learns without labels. However, large-scale unlabeled datasets in the wild often have long-tailed label distributions, where we know little about the behavior of SSL. In this work, we systematically investigate self-supervised learning under dataset imbalance. First, we find out via extensive experiments that off-the-shelf self-supervised representations are already more robust to class imbalance than supervised representations. The performance gap between balanced and imbalanced pre-training with SSL is significantly smaller than the gap with supervised learning, across sample sizes, for both in-domain and, especially, out-of-domain evaluation. Second, towards understanding the robustness of SSL, we hypothesize that SSL learns richer features from frequent data: it may learn label-irrelevant-but-transferable features that help classify the rare classes and downstream tasks. In contrast, supervised learning has no incentive to learn features irrelevant to the labels from frequent examples. We validate this hypothesis with semi-synthetic experiments and theoretical analyses on a simplified setting. Third, inspired by the theoretical insights, we devise a re-weighted regularization technique that consistently improves the SSL representation quality on imbalanced datasets with several evaluation criteria, closing the small gap between balanced and imbalanced datasets with the same number of examples.

preprint2022arXiv

Shape of the asymptotic maximum sum-free sets in integer lattice grids

We determine the shape of all sum-free sets in $\{1,2,\ldots,n\}^2$ of size close to the maximum $\frac{3}{5}n^2$, solving a problem of Elsholtz and Rackham. We show that all such asymptotic maximum sum-free sets lie completely in the stripe $\frac{4}{5}n-o(n)\le x+y\le\frac{8}{5}n+ o(n)$. We also determine for any positive integer $p$ the maximum size of a subset $A\subseteq \{1,2,\ldots,n\}^2$ which forbids the triple $(x,y,z)$ satisfying $px+py=z$.

preprint2022arXiv

Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retina OCT Images

Automated surface segmentation of retinal layer is important and challenging in analyzing optical coherence tomography (OCT). Recently, many deep learning based methods have been developed for this task and yield remarkable performance. However, due to large spatial gap and potential mismatch between the B-scans of OCT data, all of them are based on 2D segmentation of individual B-scans, which may loss the continuity information across the B-scans. In addition, 3D surface of the retina layers can provide more diagnostic information, which is crucial in quantitative image analysis. In this study, a novel framework based on hybrid 2D-3D convolutional neural networks (CNNs) is proposed to obtain continuous 3D retinal layer surfaces from OCT. The 2D features of individual B-scans are extracted by an encoder consisting of 2D convolutions. These 2D features are then used to produce the alignment displacement field and layer segmentation by two 3D decoders, which are coupled via a spatial transformer module. The entire framework is trained end-to-end. To the best of our knowledge, this is the first study that attempts 3D retinal layer segmentation in volumetric OCT images based on CNNs. Experiments on a publicly available dataset show that our framework achieves superior results to state-of-the-art 2D methods in terms of both layer segmentation accuracy and cross-B-scan 3D continuity, thus offering more clinical values than previous works.

preprint2022arXiv

Snowmass White Paper: Effective Field Theories for Condensed Matter Systems

We review recent progress and a number of future directions for applications of effective field theory methods to condensed matter systems broadly defined. Our emphasis is on areas that have allowed a fertile exchange of ideas between high energy physics and many-body theory. We discuss developments in the effective field theory of spontaneous symmetry breaking, of hydrodynamics and non-equilibrium dynamics more generally, fracton phases of matter, and dualities between 2+1 dimensional field theories. We furthermore discuss the application of effective field theory to non-Fermi liquids, the dynamics of entanglement entropy and to condensed matter aspects of cosmology.

preprint2022arXiv

Snowmass White Paper: New ideas for many-body quantum systems from string theory and black holes

During the last two decades many new insights into the dynamics of strongly coupled quantum many-body systems have been obtained using gauge/gravity duality, with black holes often playing a universal role. In this white paper we summarize the results obtained and offer some outlook for future developments, including the ongoing mutually beneficial feedback loop with the study of more general, not necessarily holographic, quantum many-body systems.

preprint2022arXiv

SP-SEDT: Self-supervised Pre-training for Sound Event Detection Transformer

Recently, an event-based end-to-end model (SEDT) has been proposed for sound event detection (SED) and achieves competitive performance. However, compared with the frame-based model, it requires more training data with temporal annotations to improve the localization ability. Synthetic data is an alternative, but it suffers from a great domain gap with real recordings. Inspired by the great success of UP-DETR in object detection, we propose to self-supervisedly pre-train SEDT (SP-SEDT) by detecting random patches (only cropped along the time axis). Experiments on the DCASE2019 task4 dataset show the proposed SP-SEDT can outperform fine-tuned frame-based model. The ablation study is also conducted to investigate the impact of different loss functions and patch size.

preprint2022arXiv

Spatiotemporal Propagation Learning for Network-Wide Flight Delay Prediction

Demystifying the delay propagation mechanisms among multiple airports is fundamental to precise and interpretable delay prediction, which is crucial during decision-making for all aviation industry stakeholders. The principal challenge lies in effectively leveraging the spatiotemporal dependencies and exogenous factors related to the delay propagation. However, previous works only consider limited spatiotemporal patterns with few factors. To promote more comprehensive propagation modeling for delay prediction, we propose SpatioTemporal Propagation Network (STPN), a space-time separable graph convolutional network, which is novel in spatiotemporal dependency capturing. From the aspect of spatial relation modeling, we propose a multi-graph convolution model considering both geographic proximity and airline schedule. From the aspect of temporal dependency capturing, we propose a multi-head self-attentional mechanism that can be learned end-to-end and explicitly reason multiple kinds of temporal dependency of delay time series. We show that the joint spatial and temporal learning models yield a sum of the Kronecker product, which factors the spatiotemporal dependence into the sum of several spatial and temporal adjacency matrices. By this means, STPN allows cross-talk of spatial and temporal factors for modeling delay propagation. Furthermore, a squeeze and excitation module is added to each layer of STPN to boost meaningful spatiotemporal features. To this end, we apply STPN to multi-step ahead arrival and departure delay prediction in large-scale airport networks. To validate the effectiveness of our model, we experiment with two real-world delay datasets, including U.S and China flight delays; and we show that STPN outperforms state-of-the-art methods. In addition, counterfactuals produced by STPN show that it learns explainable delay propagation patterns.

preprint2022arXiv

SRP-DNN: Learning Direct-Path Phase Difference for Multiple Moving Sound Source Localization

Multiple moving sound source localization in real-world scenarios remains a challenging issue due to interaction between sources, time-varying trajectories, distorted spatial cues, etc. In this work, we propose to use deep learning techniques to learn competing and time-varying direct-path phase differences for localizing multiple moving sound sources. A causal convolutional recurrent neural network is designed to extract the direct-path phase difference sequence from signals of each microphone pair. To avoid the assignment ambiguity and the problem of uncertain output-dimension encountered when simultaneously predicting multiple targets, the learning target is designed in a weighted sum format, which encodes source activity in the weight and direct-path phase differences in the summed value. The learned direct-path phase differences for all microphone pairs can be directly used to construct the spatial spectrum according to the formulation of steered response power (SRP). This deep neural network (DNN) based SRP method is referred to as SRP-DNN. The locations of sources are estimated by iteratively detecting and removing the dominant source from the spatial spectrum, in which way the interaction between sources is reduced. Experimental results on both simulated and real-world data show the superiority of the proposed method in the presence of noise and reverberation.

preprint2022arXiv

Transformer for Single Image Super-Resolution

Single image super-resolution (SISR) has witnessed great strides with the development of deep learning. However, most existing studies focus on building more complex networks with a massive number of layers. Recently, more and more researchers start to explore the application of Transformer in computer vision tasks. However, the heavy computational cost and high GPU memory occupation of the vision Transformer cannot be ignored. In this paper, we propose a novel Efficient Super-Resolution Transformer (ESRT) for SISR. ESRT is a hybrid model, which consists of a Lightweight CNN Backbone (LCB) and a Lightweight Transformer Backbone (LTB). Among them, LCB can dynamically adjust the size of the feature map to extract deep features with a low computational cost. LTB is composed of a series of Efficient Transformers (ET), which occupies a small GPU memory occupation, thanks to the specially designed Efficient Multi-Head Attention (EMHA). Extensive experiments show that ESRT achieves competitive results with low computational costs. Compared with the original Transformer which occupies 16,057M GPU memory, ESRT only occupies 4,191M GPU memory. All codes are available at https://github.com/luissen/ESRT.

preprint2022arXiv

Transformers Improve Breast Cancer Diagnosis from Unregistered Multi-View Mammograms

Deep convolutional neural networks (CNNs) have been widely used in various medical imaging tasks. However, due to the intrinsic locality of convolution operation, CNNs generally cannot model long-range dependencies well, which are important for accurately identifying or mapping corresponding breast lesion features computed from unregistered multiple mammograms. This motivates us to leverage the architecture of Multi-view Vision Transformers to capture long-range relationships of multiple mammograms from the same patient in one examination. For this purpose, we employ local Transformer blocks to separately learn patch relationships within four mammograms acquired from two-view (CC/MLO) of two-side (right/left) breasts. The outputs from different views and sides are concatenated and fed into global Transformer blocks, to jointly learn patch relationships between four images representing two different views of the left and right breasts. To evaluate the proposed model, we retrospectively assembled a dataset involving 949 sets of mammograms, which include 470 malignant cases and 479 normal or benign cases. We trained and evaluated the model using a five-fold cross-validation method. Without any arduous preprocessing steps (e.g., optimal window cropping, chest wall or pectoral muscle removal, two-view image registration, etc.), our four-image (two-view-two-side) Transformer-based model achieves case classification performance with an area under ROC curve (AUC = 0.818), which significantly outperforms AUC = 0.784 achieved by the state-of-the-art multi-view CNNs (p = 0.009). It also outperforms two one-view-two-side models that achieve AUC of 0.724 (CC view) and 0.769 (MLO view), respectively. The study demonstrates the potential of using Transformers to develop high-performing computer-aided diagnosis schemes that combine four mammograms.

preprint2022arXiv

Virtual Adversarial Training for Semi-supervised Breast Mass Classification

This study aims to develop a novel computer-aided diagnosis (CAD) scheme for mammographic breast mass classification using semi-supervised learning. Although supervised deep learning has achieved huge success across various medical image analysis tasks, its success relies on large amounts of high-quality annotations, which can be challenging to acquire in practice. To overcome this limitation, we propose employing a semi-supervised method, i.e., virtual adversarial training (VAT), to leverage and learn useful information underlying in unlabeled data for better classification of breast masses. Accordingly, our VAT-based models have two types of losses, namely supervised and virtual adversarial losses. The former loss acts as in supervised classification, while the latter loss aims at enhancing model robustness against virtual adversarial perturbation, thus improving model generalizability. To evaluate the performance of our VAT-based CAD scheme, we retrospectively assembled a total of 1024 breast mass images, with equal number of benign and malignant masses. A large CNN and a small CNN were used in this investigation, and both were trained with and without the adversarial loss. When the labeled ratios were 40% and 80%, VAT-based CNNs delivered the highest classification accuracy of 0.740 and 0.760, respectively. The experimental results suggest that the VAT-based CAD scheme can effectively utilize meaningful knowledge from unlabeled data to better classify mammographic breast mass images.

preprint2022arXiv

Weakly-supervised 3D Human Pose Estimation with Cross-view U-shaped Graph Convolutional Network

Although monocular 3D human pose estimation methods have made significant progress, it is far from being solved due to the inherent depth ambiguity. Instead, exploiting multi-view information is a practical way to achieve absolute 3D human pose estimation. In this paper, we propose a simple yet effective pipeline for weakly-supervised cross-view 3D human pose estimation. By only using two camera views, our method can achieve state-of-the-art performance in a weakly-supervised manner, requiring no 3D ground truth but only 2D annotations. Specifically, our method contains two steps: triangulation and refinement. First, given the 2D keypoints that can be obtained through any classic 2D detection methods, triangulation is performed across two views to lift the 2D keypoints into coarse 3D poses. Then, a novel cross-view U-shaped graph convolutional network (CV-UGCN), which can explore the spatial configurations and cross-view correlations, is designed to refine the coarse 3D poses. In particular, the refinement progress is achieved through weakly-supervised learning, in which geometric and structure-aware consistency checks are performed. We evaluate our method on the standard benchmark dataset, Human3.6M. The Mean Per Joint Position Error on the benchmark dataset is 27.4 mm, which outperforms existing state-of-the-art methods remarkably (27.4 mm vs 30.2 mm).

preprint2021arXiv

Bound entanglement in thermalized states and black hole radiation

We study the mixed-state entanglement structure of chaotic quantum many-body systems at late times using the recently developed $\textit{equilibrium approximation}$. A rich entanglement phase diagram emerges when we generalize this technique to evaluate the logarithmic negativity for various universality classes of macroscopically thermalized states. Unlike in the infinite temperature case, when we impose energy constraints at finite temperature, the phase diagrams for the logarithmic negativity and the mutual information become distinct. In particular, we identify a regime where the negativity is extensive but the mutual information is sub-extensive, indicating a large amount of $\textit{bound entanglement}$. When applied to evaporating black holes, these results imply that there is quantum entanglement within the Hawking radiation long before the Page time, although this entanglement may not be distillable into EPR pairs.

preprint2021arXiv

Mixed-state entanglement and information recovery in thermalized states and evaporating black holes

We study the universal behavior of quantum information-theoretic quantities in thermalized isolated quantum many-body systems and evaporating black holes. In particular, we study a genuine mixed-state entanglement measure called the logarithmic negativity, other correlation measures including the Renyi negativities and the mutual information, and a signature of multipartite entanglement called the reflected entropy. We also probe the feasibility of recovering quantum information from subsystems of a thermalized quantum many-body system or from the radiation of an evaporating black hole, using quantities such as relative entropy and Petz map fidelity. A recently developed technique called the equilibrium approximation allows us to probe these quantities at finite temperature. We find striking qualitative differences from the infinite temperature case, which has been the topic of previous studies using Haar-random states. In particular, we find regimes where the logarithmic negativity is extensive but the mutual information is sub-extensive, indicating a large amount of undistillable, bound entanglement in thermalized states. For evaporating black holes at finite temperature, both the logarithmic negativity and the Petz map fidelity reveal an important new time scale $t_b$, which is earlier than the Page time $t_p$ by a finite fraction of the total evaporation time. We find that $t_b$, as opposed to $t_p$, is the time scale at which quantum entanglement between different parts of the radiation becomes extensive, and the fidelity of information recovery for a large diary thrown into the black hole starts to grow.

preprint2021arXiv

On systems of maximal quantum chaos

A remarkable feature of chaos in many-body quantum systems is the existence of a bound on the quantum Lyapunov exponent. An important question is to understand what is special about maximally chaotic systems which saturate this bound. Here we provide further evidence for the `hydrodynamic' origin of chaos in such systems, and discuss hallmarks of maximally chaotic systems. We first provide evidence that a hydrodynamic effective field theory of chaos we previously proposed should be understood as a theory of maximally chaotic systems. We then emphasize and make explicit a signature of maximal chaos which was only implicit in prior literature, namely the suppression of exponential growth in commutator squares of generic few-body operators. We provide a general argument for this suppression within our chaos effective field theory, and illustrate it using SYK models and holographic systems. We speculate that this suppression indicates that the nature of operator scrambling in maximally chaotic systems is fundamentally different to scrambling in non-maximally chaotic systems. We also discuss a simplest scenario for the existence of a maximally chaotic regime at sufficiently large distances even for non-maximally chaotic systems.

preprint2020arXiv

Anti-Bandit Neural Architecture Search for Model Defense

Deep convolutional neural networks (DCNNs) have dominated as the best performers in machine learning, but can be challenged by adversarial attacks. In this paper, we defend against adversarial attacks using neural architecture search (NAS) which is based on a comprehensive search of denoising blocks, weight-free operations, Gabor filters and convolutions. The resulting anti-bandit NAS (ABanditNAS) incorporates a new operation evaluation measure and search process based on the lower and upper confidence bounds (LCB and UCB). Unlike the conventional bandit algorithm using UCB for evaluation only, we use UCB to abandon arms for search efficiency and LCB for a fair competition between arms. Extensive experiments demonstrate that ABanditNAS is faster than other NAS methods, while achieving an $8.73\%$ improvement over prior arts on CIFAR-10 under PGD-$7$.

preprint2020arXiv

Applying a random projection algorithm to optimize machine learning model for breast lesion classification

Machine learning is widely used in developing computer-aided diagnosis (CAD) schemes of medical images. However, CAD usually computes large number of image features from the targeted regions, which creates a challenge of how to identify a small and optimal feature vector to build robust machine learning models. In this study, we investigate feasibility of applying a random projection algorithm to build an optimal feature vector from the initially CAD-generated large feature pool and improve performance of machine learning model. We assemble a retrospective dataset involving 1,487 cases of mammograms in which 644 cases have confirmed malignant mass lesions and 843 have benign lesions. A CAD scheme is first applied to segment mass regions and initially compute 181 features. Then, support vector machine (SVM) models embedded with several feature dimensionality reduction methods are built to predict likelihood of lesions being malignant. All SVM models are trained and tested using a leave-one-case-out cross-validation method. SVM generates a likelihood score of each segmented mass region depicting on one-view mammogram. By fusion of two scores of the same mass depicting on two-view mammograms, a case-based likelihood score is also evaluated. Comparing with the principle component analyses, nonnegative matrix factorization, and Chi-squared methods, SVM embedded with the random projection algorithm yielded a significantly higher case-based lesion classification performance with the area under ROC curve of 0.84+0.01 (p<0.02). The study demonstrates that the random project algorithm is a promising method to generate optimal feature vectors to help improve performance of machine learning models of medical images.

preprint2020arXiv

Bi-directional Exponential Angular Triplet Loss for RGB-Infrared Person Re-Identification

RGB-Infrared person re-identification (RGB-IR Re- ID) is a cross-modality matching problem, where the modality discrepancy is a big challenge. Most existing works use Euclidean metric based constraints to resolve the discrepancy between features of images from different modalities. However, these methods are incapable of learning angularly discriminative feature embedding because Euclidean distance cannot measure the included angle between embedding vectors effectively. As an angularly discriminative feature space is important for classifying the human images based on their embedding vectors, in this paper, we propose a novel ranking loss function, named Bi-directional Exponential Angular Triplet Loss, to help learn an angularly separable common feature space by explicitly constraining the included angles between embedding vectors. Moreover, to help stabilize and learn the magnitudes of embedding vectors, we adopt a common space batch normalization layer. The quantitative and qualitative experiments on the SYSU-MM01 and RegDB dataset support our analysis. On SYSU-MM01 dataset, the performance is improved from 7.40% / 11.46% to 38.57% / 38.61% for rank-1 accuracy / mAP compared with the baseline. The proposed method can be generalized to the task of single-modality Re-ID and improves the rank-1 accuracy / mAP from 92.0% / 81.7% to 94.7% / 86.6% on the Market-1501 dataset, from 82.6% / 70.6% to 87.6% / 77.1% on the DukeMTMC-reID dataset. Code: https://github.com/prismformore/expAT

preprint2020arXiv

Dynamical Phase Transition from Nonequilibrium Dynamics of Dark Solitons

By holographic duality, we identify a novel dynamical phase transition which results from the temperature dependence of non-equilibrium dynamics of dark solitons in a superfluid.For a non-equilibrium superfluid system with an initial density of dark solitons, there exists a critical temperature $T_d$,above which the system relaxes to equilibrium by producing sound waves, while below which it goes through an intermediate phase with a finite density of vortex-antivortex pairs. In particular, as $T_d$ is approached from below, the density of vortex pairs scales as $(T_d - T)^γ$ with the critical exponent $γ= 1/2$.

preprint2020arXiv

Guided learning for weakly-labeled semi-supervised sound event detection

We propose a simple but efficient method termed Guided Learning for weakly-labeled semi-supervised sound event detection (SED). There are two sub-targets implied in weakly-labeled SED: audio tagging and boundary detection. Instead of designing a single model by considering a trade-off between the two sub-targets, we design a teacher model aiming at audio tagging to guide a student model aiming at boundary detection to learn using the unlabeled data. The guidance is guaranteed by the audio tagging performance gap of the two models. In the meantime, the student model liberated from the trade-off is able to provide more excellent boundary detection results. We propose a principle to design such two models based on the relation between the temporal compression scale and the two sub-targets. We also propose an end-to-end semi-supervised learning process for these two models to enable their abilities to rise alternately. Experiments on the DCASE2018 Task4 dataset show that our approach achieves competitive performance.

preprint2020arXiv

Hadamard Matrix Guided Online Hashing

Online image hashing has attracted increasing research attention recently, which receives large-scale data in a streaming manner to update the hash functions on-the-fly. Its key challenge lies in the difficulty of balancing the learning timeliness and model accuracy. To this end, most works follow a supervised setting, i.e., using class labels to boost the hashing performance, which defects in two aspects: First, strong constraints, e.g., orthogonal or similarity preserving, are used, which however are typically relaxed and lead to large accuracy drop. Second, large amounts of training batches are required to learn the up-to-date hash functions, which largely increase the learning complexity. To handle the above challenges, a novel supervised online hashing scheme termed Hadamard Matrix Guided Online Hashing (HMOH) is proposed in this paper. Our key innovation lies in introducing Hadamard matrix, which is an orthogonal binary matrix built via Sylvester method. In particular, to release the need of strong constraints, we regard each column of Hadamard matrix as the target code for each class label, which by nature satisfies several desired properties of hashing codes. To accelerate the online training, LSH is first adopted to align the lengths of target code and to-be-learned binary code. We then treat the learning of hash functions as a set of binary classification problems to fit the assigned target code. Finally, extensive experiments demonstrate the superior accuracy and efficiency of the proposed method over various state-of-the-art methods. Codes are available at https://github.com/lmbxmu/mycode.

preprint2020arXiv

Multi-Branch Learning for Weakly-Labeled Sound Event Detection

There are two sub-tasks implied in the weakly-supervised SED: audio tagging and event boundary detection. Current methods which combine multi-task learning with SED requires annotations both for these two sub-tasks. Since there are only annotations for audio tagging available in weakly-supervised SED, we design multiple branches with different learning purposes instead of pursuing multiple tasks. Similar to multiple tasks, multiple different learning purposes can also prevent the common feature which the multiple branches share from overfitting to any one of the learning purposes. We design these multiple different learning purposes based on combinations of different MIL strategies and different pooling methods. Experiments on the DCASE 2018 Task 4 dataset and the URBAN-SED dataset both show that our method achieves competitive performance.

preprint2020arXiv

Online Initialization and Extrinsic Spatial-Temporal Calibration for Monocular Visual-Inertial Odometry

This paper presents an online initialization method for bootstrapping the optimization-based monocular visual-inertial odometry (VIO). The method can online calibrate the relative transformation (spatial) and time offsets (temporal) among camera and IMU, as well as estimate the initial values of metric scale, velocity, gravity, gyroscope bias, and accelerometer bias during the initialization stage. To compensate for the impact of time offset, our method includes two short-term motion interpolation algorithms for the camera and IMU pose estimation. Besides, it includes a three-step process to incrementally estimate the parameters from coarse to fine. First, the extrinsic rotation, gyroscope bias, and time offset are estimated by minimizing the rotation difference between the camera and IMU. Second, the metric scale, gravity, and extrinsic translation are approximately estimated by using the compensated camera poses and ignoring the accelerometer bias. Third, these values are refined by taking into account the accelerometer bias and the gravitational magnitude. For further optimizing the system states, a nonlinear optimization algorithm, which considers the time offset, is introduced for global and local optimization. Experimental results on public datasets show that the initial values and the extrinsic parameters, as well as the sensor poses, can be accurately estimated by the proposed method.

preprint2020arXiv

Projection & Probability-Driven Black-Box Attack

Generating adversarial examples in a black-box setting retains a significant challenge with vast practical application prospects. In particular, existing black-box attacks suffer from the need for excessive queries, as it is non-trivial to find an appropriate direction to optimize in the high-dimensional space. In this paper, we propose Projection & Probability-driven Black-box Attack (PPBA) to tackle this problem by reducing the solution space and providing better optimization. For reducing the solution space, we first model the adversarial perturbation optimization problem as a process of recovering frequency-sparse perturbations with compressed sensing, under the setting that random noise in the low-frequency space is more likely to be adversarial. We then propose a simple method to construct a low-frequency constrained sensing matrix, which works as a plug-and-play projection matrix to reduce the dimensionality. Such a sensing matrix is shown to be flexible enough to be integrated into existing methods like NES and Bandits$_{TD}$. For better optimization, we perform a random walk with a probability-driven strategy, which utilizes all queries over the whole progress to make full use of the sensing matrix for a less query budget. Extensive experiments show that our method requires at most 24% fewer queries with a higher attack success rate compared with state-of-the-art approaches. Finally, the attack method is evaluated on the real-world online service, i.e., Google Cloud Vision API, which further demonstrates our practical potentials.

preprint2020arXiv

Quantum many-body physics from a gravitational lens

The last two decades have seen the emergence of stunning interconnections among various previously remotely related disciplines such as condensed matter, nuclear physics, gravity and quantum information, fueled both by experimental advances and new powerful theoretical methods brought by holographic duality. In this non-technical review we sample some recent developments in holographic duality in connection with quantum many-body dynamics. These include insights into strongly correlated phases without quasiparticles and their transport properties, quantum many-body chaos, and scrambling of quantum information. We also discuss recent progress in understanding the structure of holographic duality itself using quantum information, including a "local" version of the duality as well as the quantum error correction interpretation of quantum many-body states with a gravity dual, and how such notions help demonstrate the unitarity of black hole evaporation.

preprint2020arXiv

Self-Refining Deep Symmetry Enhanced Network for Rain Removal

Rain removal aims to remove the rain streaks on rain images. The state-of-the-art methods are mostly based on Convolutional Neural Network~(CNN). However, as CNN is not equivariant to object rotation, these methods are unsuitable for dealing with the tilted rain streaks. To tackle this problem, we propose Deep Symmetry Enhanced Network~(DSEN) that is able to explicitly extract the rotation equivariant features from rain images. In addition, we design a self-refining mechanism to remove the accumulated rain streaks in a coarse-to-fine manner. This mechanism reuses DSEN with a novel information link which passes the gradient flow to the higher stages. Extensive experiments on both synthetic and real-world rain images show that our self-refining DSEN yields the top performance.

preprint2020arXiv

Spatial Pyramid Based Graph Reasoning for Semantic Segmentation

The convolution operation suffers from a limited receptive filed, while global modeling is fundamental to dense prediction tasks, such as semantic segmentation. In this paper, we apply graph convolution into the semantic segmentation task and propose an improved Laplacian. The graph reasoning is directly performed in the original feature space organized as a spatial pyramid. Different from existing methods, our Laplacian is data-dependent and we introduce an attention diagonal matrix to learn a better distance metric. It gets rid of projecting and re-projecting processes, which makes our proposed method a light-weight module that can be easily plugged into current computer vision architectures. More importantly, performing graph reasoning directly in the feature space retains spatial relationships and makes spatial pyramid possible to explore multiple long-range contextual patterns from different scales. Experiments on Cityscapes, COCO Stuff, PASCAL Context and PASCAL VOC demonstrate the effectiveness of our proposed methods on semantic segmentation. We achieve comparable performance with advantages in computational and memory overhead.

preprint2020arXiv

Specialized Decision Surface and Disentangled Feature for Weakly-Supervised Polyphonic Sound Event Detection

In this paper, a special decision surface for the weakly-supervised sound event detection (SED) and a disentangled feature (DF) for the multi-label problem in polyphonic SED are proposed. We approach SED as a multiple instance learning (MIL) problem and utilize a neural network framework with a pooling module to solve it. General MIL approaches include two kinds: the instance-level approaches and embedding-level approaches. We present a method of generating instance-level probabilities for the embedding level approaches which tend to perform better than the instance-level approaches in terms of bag-level classification but can not provide instance-level probabilities in current approaches. Moreover, we further propose a specialized decision surface (SDS) for the embedding-level attention pooling. We analyze and explained why an embedding-level attention module with SDS is better than other typical pooling modules from the perspective of the high-level feature space. As for the problem of the unbalanced dataset and the co-occurrence of multiple categories in the polyphonic event detection task, we propose a DF to reduce interference among categories, which optimizes the high-level feature space by disentangling it based on class-wise identifiable information and obtaining multiple different subspaces. Experiments on the dataset of DCASE 2018 Task 4 show that the proposed SDS and DF significantly improve the detection performance of the embedding-level MIL approach with an attention pooling module and outperform the first place system in the challenge by 6.6 percentage points.

preprint2020arXiv

The exact minimum number of triangles in graphs of given order and size

What is the minimum number of triangles in a graph of given order and size? Motivated by earlier results of Mantel and Turán, Rademacher solved the first non-trivial case of this problem in 1941. The problem was revived by Erdős in 1955; it is now known as the Erdős-Rademacher problem. After attracting much attention, it was solved asymptotically in a major breakthrough by Razborov in 2008. In this paper, we provide an exact solution for all large graphs whose edge density is bounded away from~$1$, which in this range confirms a conjecture of Lovász and Simonovits from 1975. Furthermore, we give a description of the extremal graphs.

preprint2020arXiv

Video Logo Retrieval based on local Features

Estimation of the frequency and duration of logos in videos is important and challenging in the advertisement industry as a way of estimating the impact of ad purchases. Since logos occupy only a small area in the videos, the popular methods of image retrieval could fail. This paper develops an algorithm called Video Logo Retrieval (VLR), which is an image-to-video retrieval algorithm based on the spatial distribution of local image descriptors that measure the distance between the query image (the logo) and a collection of video images. VLR uses local features to overcome the weakness of global feature-based models such as convolutional neural networks (CNN). Meanwhile, VLR is flexible and does not require training after setting some hyper-parameters. The performance of VLR is evaluated on two challenging open benchmark tasks (SoccerNet and Standford I2V), and compared with other state-of-the-art logo retrieval or detection algorithms. Overall, VLR shows significantly higher accuracy compared with the existing methods.

preprint2020arXiv

When Dictionary Learning Meets Deep Learning: Deep Dictionary Learning and Coding Network for Image Recognition with Limited Data

We present a new Deep Dictionary Learning and Coding Network (DDLCN) for image recognition tasks with limited data. The proposed DDLCN has most of the standard deep learning layers (e.g., input/output, pooling, fully connected, etc.), but the fundamental convolutional layers are replaced by our proposed compound dictionary learning and coding layers. The dictionary learning learns an over-complete dictionary for input training data. At the deep coding layer, a locality constraint is added to guarantee that the activated dictionary bases are close to each other. Then the activated dictionary atoms are assembled and passed to the compound dictionary learning and coding layers. In this way, the activated atoms in the first layer can be represented by the deeper atoms in the second dictionary. Intuitively, the second dictionary is designed to learn the fine-grained components shared among the input dictionary atoms, thus a more informative and discriminative low-level representation of the dictionary atoms can be obtained. We empirically compare DDLCN with several leading dictionary learning methods and deep learning models. Experimental results on five popular datasets show that DDLCN achieves competitive results compared with state-of-the-art methods when the training data is limited. Code is available at https://github.com/Ha0Tang/DDLCN.

preprint2020arXiv

WQT and DG-YOLO: towards domain generalization in underwater object detection

A General Underwater Object Detector (GUOD) should perform well on most of underwater circumstances. However, with limited underwater dataset, conventional object detection methods suffer from domain shift severely. This paper aims to build a GUOD with small underwater dataset with limited types of water quality. First, we propose a data augmentation method Water Quality Transfer (WQT) to increase domain diversity of the original small dataset. Second, for mining the semantic information from data generated by WQT, DG-YOLO is proposed, which consists of three parts: YOLOv3, DIM and IRM penalty. Finally, experiments on original and synthetic URPC2019 dataset prove that WQT+DG-YOLO achieves promising performance of domain generalization in underwater object detection.

preprint2019arXiv

Compact all-fiber polarization-independent up-conversion single-photon detector

We demonstrate a compact all-fiber polarization-independent up-conversion single-photon detector based on integrated reverse proton exchanged periodically poled lithium niobate waveguides. The horizontally and vertically polarized components of randomly polarized signals are separated with a fiber-coupled polarization beam splitter, launched into two orthogonally polarized polarization maintaining fibers and fetched into two adjacent independent waveguides on the same device. The up-converted outputs from both waveguide channels are then combined with a multi-mode fiber combiner and detected by a silicon detector. With this configuration, the polarization-independent single-photon counting at 1.55 um is achieved with a system detection efficiency of 29.3%, a dark count rate of 1600 counts per second, and a polarization dependent loss of 0.1dB. This compact all-fiber system is robust and has great application potential in practical quantum key distribution systems.

preprint2019arXiv

Sample phase gradient and fringe phase shift in dual phase grating X-ray interferometry

One of the key tasks in grating based x-ray phase contrast imaging is to accurately retrieve local phase gradients of a sample from measured intensity fringe shifts. To fulfill this task in dual phase grating interferometry, one needs to know the exact mathematical relationship between the two. In this work, using intuitive analysis of the sample-generated fringe shifts based on the beat pattern formation mechanism, the authors derived the formulas relating sample's phase gradients to fringe phase shifts. These formulas provide also a design optimization tool for dual phase grating interferometry.

preprint2019arXiv

Tungsten Boride: a 2D Multiple Dirac Semimetal for Hydrogen Evolution Reaction

Here, we propose a two-dimensional tungsten boride (WB4) lattice, with the Gibbs free energy for the adsorption of atomic hydrogen, tending to be the ideal value of 0 eV at 3% strained state, to host a better hydrogen evolution reaction activity. Based on first-principles calculations, it is demonstrated that the multiple d-p-pi and d-p-sigma Dirac conjugations of WB4 lattice ensures its excellent electronic transport characteristics. Meanwhile, coupling with the d-orbitals of W, the p-orbitals of borophene subunits in WB4 lattice can modulate the d band center to get a good HER performance. Our results not only provide a versatile platform for hosting multiple Dirac semimetal states with a sandwich configuration, but also offer a guiding principle for discovering the relationship between intrinsic properties of the active centre and the catalytic activity of metal layer from the emerging field of low-dimensional noble-metal-free lattices.

Hong Liu

Quick read

Decide how to stay connected

How to connect with this researcher

Open a focused conversation when the fit is right

See the researcher in context

Building this graph slice

61 published item(s)

MoPO: Incorporating Motion Prior for Occluded Human Mesh Recovery

A Gated Cross-domain Collaborative Network for Underwater Object Detection

A systematic formulation of chiral anomalous magnetohydrodynamics

Achieving Domain Generalization in Underwater Object Detection by Domain Mixup and Contrastive Learning

Many Hamiltonian subsets in large graphs with given density

Polynomial Schur&#39;s theorem

Advancing Semi-Supervised Task Oriented Dialog Systems by JSA Learning of Discrete Latent Variable Models

AO2-DETR: Arbitrary-Oriented Object Detection Transformer

Breast Cancer Molecular Subtypes Prediction on Pathological Images with Discriminative Patch Selecting and Multi-Instance Learning

Clique immersion in graphs without fixed bipartite graph

Contrastive Learning from Spatio-Temporal Mixed Skeleton Sequences for Self-Supervised Skeleton-Based Action Recognition

Disjoint isomorphic balanced clique subdivisions

Exploiting Temporal Contexts with Strided Transformer for 3D Human Pose Estimation

Exponential decay of intersection volume with applications on list-decodability and Gilbert-Varshamov type bound

Facial-Sketch Synthesis: A New Challenge

High-order Photonic Cavity Modes Enabled 3D Structural Color

How to build a pillar: a proof of Thomassen&#39;s conjecture

Identity-Sensitive Knowledge Propagation for Cloth-Changing Person Re-identification

Learning Deep Direct-Path Relative Transfer Function for Binaural Sound Source Localization

MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimation

Mission Apollo: Landing Optical Circuit Switching at Datacenter Scale

Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognition

PetLock:A Genderless and Standard Interface for the Future On-orbit Construction

Prototype Design and Efficiency Analysis of a Novel Robot Drive Based on 3K-H-V Topology

Recent advances and clinical applications of deep learning in medical image analysis

Recent Advances in Tunable Metasurfaces: Materials, Design and Applications

Self-supervised Learning is More Robust to Dataset Imbalance

Shape of the asymptotic maximum sum-free sets in integer lattice grids

Simultaneous Alignment and Surface Regression Using Hybrid 2D-3D Networks for 3D Coherent Layer Segmentation of Retina OCT Images

Snowmass White Paper: Effective Field Theories for Condensed Matter Systems

Snowmass White Paper: New ideas for many-body quantum systems from string theory and black holes

SP-SEDT: Self-supervised Pre-training for Sound Event Detection Transformer

Spatiotemporal Propagation Learning for Network-Wide Flight Delay Prediction

SRP-DNN: Learning Direct-Path Phase Difference for Multiple Moving Sound Source Localization

Transformer for Single Image Super-Resolution

Transformers Improve Breast Cancer Diagnosis from Unregistered Multi-View Mammograms

Virtual Adversarial Training for Semi-supervised Breast Mass Classification

Weakly-supervised 3D Human Pose Estimation with Cross-view U-shaped Graph Convolutional Network

Bound entanglement in thermalized states and black hole radiation

Mixed-state entanglement and information recovery in thermalized states and evaporating black holes

On systems of maximal quantum chaos

Anti-Bandit Neural Architecture Search for Model Defense

Applying a random projection algorithm to optimize machine learning model for breast lesion classification

Bi-directional Exponential Angular Triplet Loss for RGB-Infrared Person Re-Identification

Dynamical Phase Transition from Nonequilibrium Dynamics of Dark Solitons

Guided learning for weakly-labeled semi-supervised sound event detection

Hadamard Matrix Guided Online Hashing

Multi-Branch Learning for Weakly-Labeled Sound Event Detection

Online Initialization and Extrinsic Spatial-Temporal Calibration for Monocular Visual-Inertial Odometry

Projection & Probability-Driven Black-Box Attack

Quantum many-body physics from a gravitational lens

Self-Refining Deep Symmetry Enhanced Network for Rain Removal

Spatial Pyramid Based Graph Reasoning for Semantic Segmentation

Specialized Decision Surface and Disentangled Feature for Weakly-Supervised Polyphonic Sound Event Detection

The exact minimum number of triangles in graphs of given order and size

Video Logo Retrieval based on local Features

When Dictionary Learning Meets Deep Learning: Deep Dictionary Learning and Coding Network for Image Recognition with Limited Data

WQT and DG-YOLO: towards domain generalization in underwater object detection

Compact all-fiber polarization-independent up-conversion single-photon detector

Sample phase gradient and fringe phase shift in dual phase grating X-ray interferometry

Tungsten Boride: a 2D Multiple Dirac Semimetal for Hydrogen Evolution Reaction

Polynomial Schur's theorem

How to build a pillar: a proof of Thomassen's conjecture