BZPEERReading, trust and collaboration for research
Menu

Discover

GraphFollow connections around a paper, topic or saved view.

Workspaces

HomePick up where your research flow left off.InboxHandle requests, replies and notifications from one place.CollectionsCurate reading lists, evidence sets and shareable dossiers.ResearchSee your private provenance graph, trail and imports.CollaborationMove from discovery into focused discussion rooms.PublishDraft, preview and publish full-text research articles.

Network

ExpertsFind specialists worth following or asking for help.CommunitiesJoin research circles organized around shared work.PartnersSee external organizations active around the same topics.

Opportunities

ListingsBrowse roles, calls and collaborations with clearer fit signals.

Account

Log inReturn to your reading and collaboration workspace.Create accountStart saving work, following people and joining teams.
Log inCreate account

Researcher profile

Ning Xu

Ning Xu contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate
Computer VisionMachine Learningcond-mat.softArtificial IntelligenceComputation and Languageeess.IVmath.CVcond-mat.mes-hallcond-mat.mtrl-scicond-mat.stat-mechcond-mat.supr-conphysics.opticsRobotics

Trust snapshot

Quick read

Trust 21 - EmergingVerification L1Unclaimed author
35works
0followers
13topics
4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Identity and collaboration

How to connect with this researcher

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Log in to claim

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Research graph

See the researcher in context

Open full explorer

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

35 published item(s)

preprint2026arXiv

Beyond "I cannot fulfill this request": Alleviating Rigid Rejection in LLMs via Label Enhancement

Large Language Models (LLMs) rely on safety alignment to obey safe requests while refusing harmful ones. However, traditional refusal mechanisms often lead to "rigid rejection," where a general template (e.g., "I cannot fulfill this request") indiscriminately triggers refusals and severely undermines the naturalness of interactions between humans and LLMs. To address this issue, LANCE is proposed in this paper to ensure safe yet flexible and natural responses via label enhancement. Specifically, LANCE employs variational inference to perform label enhancement, predicting a continuous distribution across multiple rejection categories. These fine-grained rejection distributions provide multi-way textual gradients for a refinement model to neutralize the hazardous elements in the prompt, so that the LLMs could generate safe responses that avoid rigid rejections while preserving the naturalness of interactions. Experiments demonstrate that LANCE significantly alleviates the rigid rejection problem while maintaining high security standards, significantly outperforming existing baseline models in terms of helpfulness and naturalness of responses.

0 citations0 reviewsNo code linkedOpen access
preprint2026arXiv

Towards Understanding Feature Learning in Parameter Transfer

Parameter transfer is a central paradigm in transfer learning, enabling knowledge reuse across tasks and domains by sharing model parameters between upstream and downstream models. However, when only a subset of parameters from the upstream model is transferred to the downstream model, there remains a lack of theoretical understanding of the conditions under which such partial parameter reuse is beneficial and of the factors that govern its effectiveness. To address this gap, we analyze a setting in which both the upstream and downstream models are ReLU convolutional neural networks (CNNs). Within this theoretical framework, we characterize how the inherited parameters act as carriers of universal knowledge and identify key factors that amplify their beneficial impact on the target task. Furthermore, our analysis provides insight into why, in certain cases, transferring parameters can lead to lower test accuracy on the target task than training a new model from scratch. To our best knowledge, our theory is the first to provide a dynamic analysis for parameter transfer and also the first to prove the existence of negative transfer theoretically. Numerical experiments and real-world data experiments are conducted to empirically validate our theoretical findings.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Training

Recent image inpainting methods have made great progress but often struggle to generate plausible image structures when dealing with large holes in complex images. This is partially due to the lack of effective network structures that can capture both the long-range dependency and high-level semantics of an image. We propose cascaded modulation GAN (CM-GAN), a new network design consisting of an encoder with Fourier convolution blocks that extract multi-scale feature representations from the input image with holes and a dual-stream decoder with a novel cascaded global-spatial modulation block at each scale level. In each decoder block, global modulation is first applied to perform coarse and semantic-aware structure synthesis, followed by spatial modulation to further adjust the feature map in a spatially adaptive fashion. In addition, we design an object-aware training scheme to prevent the network from hallucinating new objects inside holes, fulfilling the needs of object removal tasks in real-world scenarios. Extensive experiments are conducted to show that our method significantly outperforms existing methods in both quantitative and qualitative evaluation. Please refer to the project page: \url{https://github.com/htzheng/CM-GAN-Inpainting}.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

CVM-Cervix: A Hybrid Cervical Pap-Smear Image Classification Framework Using CNN, Visual Transformer and Multilayer Perceptron

Cervical cancer is the seventh most common cancer among all the cancers worldwide and the fourth most common cancer among women. Cervical cytopathology image classification is an important method to diagnose cervical cancer. Manual screening of cytopathology images is time-consuming and error-prone. The emergence of the automatic computer-aided diagnosis system solves this problem. This paper proposes a framework called CVM-Cervix based on deep learning to perform cervical cell classification tasks. It can analyze pap slides quickly and accurately. CVM-Cervix first proposes a Convolutional Neural Network module and a Visual Transformer module for local and global feature extraction respectively, then a Multilayer Perceptron module is designed to fuse the local and global features for the final classification. Experimental results show the effectiveness and potential of the proposed CVM-Cervix in the field of cervical Pap smear image classification. In addition, according to the practical needs of clinical work, we perform a lightweight post-processing to compress the model.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

EBHI:A New Enteroscope Biopsy Histopathological H&E Image Dataset for Image Classification Evaluation

Background and purpose: Colorectal cancer has become the third most common cancer worldwide, accounting for approximately 10% of cancer patients. Early detection of the disease is important for the treatment of colorectal cancer patients. Histopathological examination is the gold standard for screening colorectal cancer. However, the current lack of histopathological image datasets of colorectal cancer, especially enteroscope biopsies, hinders the accurate evaluation of computer-aided diagnosis techniques. Methods: A new publicly available Enteroscope Biopsy Histopathological H&E Image Dataset (EBHI) is published in this paper. To demonstrate the effectiveness of the EBHI dataset, we have utilized several machine learning, convolutional neural networks and novel transformer-based classifiers for experimentation and evaluation, using an image with a magnification of 200x. Results: Experimental results show that the deep learning method performs well on the EBHI dataset. Traditional machine learning methods achieve maximum accuracy of 76.02% and deep learning method achieves a maximum accuracy of 95.37%. Conclusion: To the best of our knowledge, EBHI is the first publicly available colorectal histopathology enteroscope biopsy dataset with four magnifications and five types of images of tumor differentiation stages, totaling 5532 images. We believe that EBHI could attract researchers to explore new classification algorithms for the automated diagnosis of colorectal cancer, which could help physicians and patients in clinical settings.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

EMDS-6: Environmental Microorganism Image Dataset Sixth Version for Image Denoising, Segmentation, Feature Extraction, Classification and Detection Methods Evaluation

Environmental microorganisms (EMs) are ubiquitous around us and have an important impact on the survival and development of human society. However, the high standards and strict requirements for the preparation of environmental microorganism (EM) data have led to the insufficient of existing related databases, not to mention the databases with GT images. This problem seriously affects the progress of related experiments. Therefore, This study develops the Environmental Microorganism Dataset Sixth Version (EMDS-6), which contains 21 types of EMs. Each type of EM contains 40 original and 40 GT images, in total 1680 EM images. In this study, in order to test the effectiveness of EMDS-6. We choose the classic algorithms of image processing methods such as image denoising, image segmentation and target detection. The experimental result shows that EMDS-6 can be used to evaluate the performance of image denoising, image segmentation, image feature extraction, image classification, and object detection methods.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

End-to-end video instance segmentation via spatial-temporal graph neural networks

Video instance segmentation is a challenging task that extends image instance segmentation to the video domain. Existing methods either rely only on single-frame information for the detection and segmentation subproblems or handle tracking as a separate post-processing step, which limit their capability to fully leverage and share useful spatial-temporal information for all the subproblems. In this paper, we propose a novel graph-neural-network (GNN) based method to handle the aforementioned limitation. Specifically, graph nodes representing instance features are used for detection and segmentation while graph edges representing instance relations are used for tracking. Both inter and intra-frame information is effectively propagated and shared via graph updates and all the subproblems (i.e. detection, segmentation and tracking) are jointly optimized in an unified framework. The performance of our method shows great improvement on the YoutubeVIS validation dataset compared to existing methods and achieves 35.2% AP with a ResNet-50 backbone, operating at 22 FPS. Code is available at http://github.com/lucaswithai/visgraph.git .

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Exploring the Semi-supervised Video Object Segmentation Problem from a Cyclic Perspective

Modern video object segmentation (VOS) algorithms have achieved remarkably high performance in a sequential processing order, while most of currently prevailing pipelines still show some obvious inadequacy like accumulative error, unknown robustness or lack of proper interpretation tools. In this paper, we place the semi-supervised video object segmentation problem into a cyclic workflow and find the defects above can be collectively addressed via the inherent cyclic property of semi-supervised VOS systems. Firstly, a cyclic mechanism incorporated to the standard sequential flow can produce more consistent representations for pixel-wise correspondance. Relying on the accurate reference mask in the starting frame, we show that the error propagation problem can be mitigated. Next, a simple gradient correction module, which naturally extends the offline cyclic pipeline to an online manner, can highlight the high-frequent and detailed part of results to further improve the segmentation quality while keeping feasible computation cost. Meanwhile such correction can protect the network from severe performance degration resulted from interference signals. Finally we develop cycle effective receptive field (cycle-ERF) based on gradient correction process to provide a new perspective into analyzing object-specific regions of interests. We conduct comprehensive comparison and detailed analysis on challenging benchmarks of DAVIS16, DAVIS17 and Youtube-VOS, demonstrating that the cyclic mechanism is helpful to enhance segmentation quality, improve the robustness of VOS systems, and further provide qualitative comparison and interpretation on how different VOS algorithms work. The code of this project can be found at https://github.com/lyxok1/STM-Training

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

High-fidelity far-field microscopy at λ/8 resolution

The emergence of far-field super-resolution microscopy has rejuvenated the possibility for nanoscale imaging. Approaches to far-field super-resolution that utilize point scanning often depends on spatially reducing the size of the focused spot. However, the focused spot always achieves high resolution at the expense of extremely low light efficiency for the probing mainlobe and high-intensity sidelobes, which limits the applications in nanoscale imaging and might cause misinterpretation of samples. Here we report a sharp probing spot with the diffraction efficiency of 3.76% at the resolution of 38% of the Airy spot size assisted by the two-dimensional multi-level diffractive optical element (DOE) experimentally. The diffraction efficiency of DOE is improved by at least two orders of magnitude at the same resolution by breaking the limitation of circular 0-π binary structure superoscillatory lens. To eliminate the influence of the high-intensity sidelobes, high-fidelity images are reconstructed based on the modified deconvolution algorithm by virtue of the prior-knowledge. Finally, high-fidelity far-field microscopy (HiFi-FM) is constructed and experimental results show that HiFi-FM allows the resolution of spatially complex samples better than 69 nm while acquiring high fidelity.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Learngene: From Open-World to Your Learning Task

Although deep learning has made significant progress on fixed large-scale datasets, it typically encounters challenges regarding improperly detecting unknown/unseen classes in the open-world scenario, over-parametrized, and overfitting small samples. Since biological systems can overcome the above difficulties very well, individuals inherit an innate gene from collective creatures that have evolved over hundreds of millions of years and then learn new skills through few examples. Inspired by this, we propose a practical collective-individual paradigm where an evolution (expandable) network is trained on sequential tasks and then recognize unknown classes in real-world. Moreover, the learngene, i.e., the gene for learning initialization rules of the target model, is proposed to inherit the meta-knowledge from the collective model and reconstruct a lightweight individual model on the target task. Particularly, a novel criterion is proposed to discover learngene in the collective model, according to the gradient information. Finally, the individual model is trained only with few samples on the target learning tasks. We demonstrate the effectiveness of our approach in an extensive empirical study and theoretical analysis.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Semantic Layout Manipulation with High-Resolution Sparse Attention

We tackle the problem of semantic image layout manipulation, which aims to manipulate an input image by editing its semantic label map. A core problem of this task is how to transfer visual details from the input images to the new semantic layout while making the resulting image visually realistic. Recent work on learning cross-domain correspondence has shown promising results for global layout transfer with dense attention-based warping. However, this method tends to lose texture details due to the resolution limitation and the lack of smoothness constraint of correspondence. To adapt this paradigm for the layout manipulation task, we propose a high-resolution sparse attention module that effectively transfers visual details to new layouts at a resolution up to 512x512. To further improve visual quality, we introduce a novel generator architecture consisting of a semantic encoder and a two-stage decoder for coarse-to-fine synthesis. Experiments on the ADE20k and Places365 datasets demonstrate that our proposed approach achieves substantial improvements over the existing inpainting and layout manipulation methods.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Solar: $L_0$ solution path averaging for fast and accurate variable selection in high-dimensional data

We propose a new variable selection algorithm, subsample-ordered least-angle regression (solar), and its coordinate descent generalization, solar-cd. Solar re-constructs lasso paths using the $L_0$ norm and averages the resulting solution paths across subsamples. Path averaging retains the ranking information of the informative variables while averaging out sensitivity to high dimensionality, improving variable selection stability, efficiency, and accuracy. We prove that: (i) with a high probability, path averaging perfectly separates informative variables from redundant variables on the average $L_0$ path; (ii) solar variable selection is consistent and accurate; and (iii) the probability that solar omits weak signals is controllable for finite sample size. We also demonstrate that: (i) solar yields, with less than $1/3$ of the lasso computation load, substantial improvements over lasso in terms of the sparsity (64-84\% reduction in redundant variable selection) and accuracy of variable selection; (ii) compared with the lasso safe/strong rule and variable screening, solar largely avoids selection of redundant variables and rejection of informative variables in the presence of complicated dependence structures; (iii) the sparsity and stability of solar conserves residual degrees of freedom for data-splitting hypothesis testing, improving the accuracy of post-selection inference on weak signals with limited $n$; (iv) replacing lasso with solar in bootstrap selection (e.g., bolasso or stability selection) produces a multi-layer variable ranking scheme that improves selection sparsity and ranking accuracy with the computation load of only one lasso realization; and (v) given the computation resources, solar bootstrap selection is substantially faster (98\% lower computation time) than the theoretical maximum speedup for parallelized bootstrap lasso (confirmed by Amdahl's law).

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translation

Remarkable achievements have been attained with Generative Adversarial Networks (GANs) in image-to-image translation. However, due to a tremendous amount of parameters, state-of-the-art GANs usually suffer from low efficiency and bulky memory usage. To tackle this challenge, firstly, this paper investigates GANs performance from a frequency perspective. The results show that GANs, especially small GANs lack the ability to generate high-quality high frequency information. To address this problem, we propose a novel knowledge distillation method referred to as wavelet knowledge distillation. Instead of directly distilling the generated images of teachers, wavelet knowledge distillation first decomposes the images into different frequency bands with discrete wavelet transformation and then only distills the high frequency bands. As a result, the student GAN can pay more attention to its learning on high frequency bands. Experiments demonstrate that our method leads to 7.08 times compression and 6.80 times acceleration on CycleGAN with almost no performance drop. Additionally, we have studied the relation between discriminators and generators which shows that the compression of discriminators can promote the performance of compressed generators.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Chiral Majorana Edge Modes and Vortex Majorana Zero Modes in Superconducting Antiferromagnetic Topological Insulator

The antiferromagnetic topological insulator (AFTI) is topologically protected by the combined time-reversal and translational symmetry $\mathcal{T}_c$. In this paper we investigate the effects of the $s$-wave superconducting pairings on the multilayers of AFTI, which breaks $\mathcal{T}_c$ symmetry and can realize quantum anomalous Hall insulator with unit Chern number. For the weakly coupled pairings, the system corresponds to the topological superconductor (TSC) with the Chern number $C=\pm 2$. We answer the following questions whether the local Chern numbers and chiral Majorana edge modes of such a TSC distribute around the surface layers. By the numerical calculations based on a theoretic model of AFTI, we find that when the local Chern numbers are always dominated by the surface layers, the wavefunctions of chiral Majorana edge modes must not localize on the surface layers and show a smooth crossover from spatially occupying all layers to only distributing near the surface layers, similar to the hinge states in a three dimensional second-order topological phases. The latter phase can be distinguished from the former phase by the measurements of the local density of state. In addition we also study the superconducting vortex phase transition in this system and find that the exchange field in the AFTI not only enlarges the phase space of topological vortex phase but also enhances its topological stability. These conclusions will stimulate the investigations on superconducting effects of AFTI and drive the studies on chiral Majorana edge modes and vortex Majorana zero modes into a new era.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Connecting glass-forming ability of binary mixtures of soft particles to equilibrium melting temperatures

The glass-forming ability is an important material property for manufacturing glasses and understanding the long-standing glass transition problem. Because of the nonequilibrium nature, it is difficult to develop the theory for it. Here we report that the glass-forming ability of binary mixtures of soft particles is related to the equilibrium melting temperatures. Due to the distinction in particle size or stiffness, the two components in a mixture effectively feel different melting temperatures, leading to a melting temperature gap. By varying the particle size, stiffness, and composition over a wide range of pressures, we establish a comprehensive picture for the glass-forming ability, based on our finding of the direct link between the glass-forming ability and the melting temperature gap. Our study reveals and explains the pressure and interaction dependence of the glass-forming ability of model glass-formers, and suggests strategies to optimize the glass-forming ability via the manipulation of particle interactions.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

High-Resolution Deep Image Matting

Image matting is a key technique for image and video editing and composition. Conventionally, deep learning approaches take the whole input image and an associated trimap to infer the alpha matte using convolutional neural networks. Such approaches set state-of-the-arts in image matting; however, they may fail in real-world matting applications due to hardware limitations, since real-world input images for matting are mostly of very high resolution. In this paper, we propose HDMatt, a first deep learning based image matting approach for high-resolution inputs. More concretely, HDMatt runs matting in a patch-based crop-and-stitch manner for high-resolution inputs with a novel module design to address the contextual dependency and consistency issues between different patches. Compared with vanilla patch-based inference which computes each patch independently, we explicitly model the cross-patch contextual dependency with a newly-proposed Cross-Patch Contextual module (CPC) guided by the given trimap. Extensive experiments demonstrate the effectiveness of the proposed method and its necessity for high-resolution inputs. Our HDMatt approach also sets new state-of-the-art performance on Adobe Image Matting and AlphaMatting benchmarks and produce impressive visual results on more real-world high-resolution images.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Jamming in confined geometry: Criticality of the jamming transition and implications of structural relaxation in confined supercooled liquids

In marginally jammed solids confined by walls, we calculate the particle and ensemble averaged value of an order parameter, $\left<Ψ(r)\right>$, as a function of the distance to the wall, $r$. Being a microscopic indicator of structural disorder and particle mobility in solids, $Ψ$ is by definition the response of the mean square particle displacement to the increase of temperature in the harmonic approximation and can be directly calculated from the normal modes of vibration of the zero-temperature solids. We find that, in confined jammed solids, $\left<Ψ(r)\right>$ curves at different pressures can collapse onto the same master curve following a scaling function, indicating the criticality of the jamming transition. The scaling collapse suggests a diverging length scale and marginal instability at the jamming transition, which should be accessible to sophisticatedly designed experiments. Moreover, $\left<Ψ(r)\right>$ is found to be significantly suppressed when approaching the wall and anisotropic in directions perpendicular and parallel to the wall. This finding can be applied to understand the $r$-dependence and anisotropy of the structural relaxation in confined supercooled liquids, providing another example of understanding or predicting behaviors of supercooled liquids from the perspective of the zero-temperature amorphous solids.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Rheological similarities between dense self-propelled and sheared particulate systems

Different from previous modelings of self-propelled particles, we develop a method to propel the particles with a constant average velocity instead of a constant force. This constant propulsion velocity (CPV) approach is validated by its agreement with the conventional constant propulsion force (CPF) approach in the flowing regime. However, the CPV approach shows its advantage of accessing quasistatic flows of yield stress fluids with a vanishing propulsion velocity, while the CPF approach is usually unable to because of finite system size. Taking this advantage, we realize the cyclic self-propulsion and study the evolution of the propulsion force with propelled particle displacement, both in the quasistatic flow regime. By mapping shear stress and shear rate to propulsion force and propulsion velocity, we find similar rheological behaviors of self-propelled systems to sheared systems, including the yield force gap between the CPF and CPV approaches, propulsion force overshoot, reversible-irreversible transition under cyclic propulsion, and propulsion bands in plastic flows. These similarities suggest the underlying connections between self-propulsion and shear, although they act on systems in different ways.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Two-scale scenario of rigidity percolation of sticky particles

In the presence of attraction, the jamming transition of packings of frictionless particles corresponds to the rigidity percolation. When the range of attraction is long, the distribution of the size of rigid clusters, $P(s)$, is continuous and shows a power-law decay. For systems with short-range attractions, however, $P(s)$ appears discontinuous. There is a power-law decay for small cluster sizes, followed by a low probability gap and a peak near the system size. We find that this appearing ``discontinuity&#39;&#39; does not mean that the transition is discontinuous. In fact, it signifies the coexistence of two distinct length scales, associated with the largest cluster and smaller ones, respectively. The comparison between the largest and second largest clusters indicates that their growth rates with system size are rather different. However, both cluster sizes tend to diverge in the large system size limit, suggesting that the jamming transition of systems with short-range attractions is still continuous. In the framework of the two-scale scenario, we also derive a generalized hyperscaling relation. With robust evidence, our work challenges the former single-scale view of the rigidity percolation.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

AOWS: Adaptive and optimal network width search with latency constraints

Neural architecture search (NAS) approaches aim at automatically finding novel CNN architectures that fit computational constraints while maintaining a good performance on the target platform. We introduce a novel efficient one-shot NAS approach to optimally search for channel numbers, given latency constraints on a specific hardware. We first show that we can use a black-box approach to estimate a realistic latency model for a specific inference platform, without the need for low-level access to the inference computation. Then, we design a pairwise MRF to score any channel configuration and use dynamic programming to efficiently decode the best performing configuration, yielding an optimal solution for the network width search. Finally, we propose an adaptive channel configuration sampling scheme to gradually specialize the training phase to the target computational constraints. Experiments on ImageNet classification show that our approach can find networks fitting the resource constraints on different target platforms while improving accuracy over the state-of-the-art efficient networks.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

CFAD: Coarse-to-Fine Action Detector for Spatiotemporal Action Localization

Most current pipelines for spatio-temporal action localization connect frame-wise or clip-wise detection results to generate action proposals, where only local information is exploited and the efficiency is hindered by dense per-frame localization. In this paper, we propose Coarse-to-Fine Action Detector (CFAD),an original end-to-end trainable framework for efficient spatio-temporal action localization. The CFAD introduces a new paradigm that first estimates coarse spatio-temporal action tubes from video streams, and then refines the tubes&#39; location based on key timestamps. This concept is implemented by two key components, the Coarse and Refine Modules in our framework. The parameterized modeling of long temporal information in the Coarse Module helps obtain accurate initial tube estimation, while the Refine Module selectively adjusts the tube location under the guidance of key timestamps. Against other methods, theproposed CFAD achieves competitive results on action detection benchmarks of UCF101-24, UCFSports and JHMDB-21 with inference speed that is 3.3x faster than the nearest competitors.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Compact Learning for Multi-Label Classification

Multi-label classification (MLC) studies the problem where each instance is associated with multiple relevant labels, which leads to the exponential growth of output space. MLC encourages a popular framework named label compression (LC) for capturing label dependency with dimension reduction. Nevertheless, most existing LC methods failed to consider the influence of the feature space or misguided by original problematic features, so that may result in performance degeneration. In this paper, we present a compact learning (CL) framework to embed the features and labels simultaneously and with mutual guidance. The proposal is a versatile concept, hence the embedding way is arbitrary and independent of the subsequent learning process. Following its spirit, a simple yet effective implementation called compact multi-label learning (CMLL) is proposed to learn a compact low-dimensional representation for both spaces. CMLL maximizes the dependence between the embedded spaces of the labels and features, and minimizes the loss of label space recovery concurrently. Theoretically, we provide a general analysis for different embedding methods. Practically, we conduct extensive experiments to validate the effectiveness of the proposed method.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Coupling between particle shape and long-range interaction in the high-density regime

By using long-range interacting polygons, we experimentally probe the coupling between particle shape and long-range interaction. For two typical space-filling polygons, square and triangle, we find two types of coupling modes that predominantly control the structure formation. Specifically, the rotational ordering of squares brings a lattice deformation that produces a hexagonal-to-rhombic transition in the high-density regime, whereas the alignment of triangles introduces a large geometric frustration that causes an order-to-disorder transition. Moreover, the two coupling modes lead to small and large &#34;internal roughness&#34; of the two systems, and thus predominantly control their structure relaxations. Our study thus provides a physical picture to the coupling between long-range interaction effect and short-range shape effect in the high-density regime unexplored before.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Essential Norms of difference of generalized composition Operators from $α$-Bloch spaces to $β$-Bloch spaces

In this paper, we study the boundedness and essential norms of the differences of two generalized composition operators acting from $α$-Bloch space to $β$-Bloch space on the open unit disk. From essential norms, we get the compactness of the differences of two generalized composition operators. This study has a relationship to the topological structure of generalized composition operators acting from $α$-Bloch space to $β$-Bloch space.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Finding Action Tubes with a Sparse-to-Dense Framework

The task of spatial-temporal action detection has attracted increasing attention among researchers. Existing dominant methods solve this problem by relying on short-term information and dense serial-wise detection on each individual frames or clips. Despite their effectiveness, these methods showed inadequate use of long-term information and are prone to inefficiency. In this paper, we propose for the first time, an efficient framework that generates action tube proposals from video streams with a single forward pass in a sparse-to-dense manner. There are two key characteristics in this framework: (1) Both long-term and short-term sampled information are explicitly utilized in our spatiotemporal network, (2) A new dynamic feature sampling module (DTS) is designed to effectively approximate the tube output while keeping the system tractable. We evaluate the efficacy of our model on the UCF101-24, JHMDB-21 and UCFSports benchmark datasets, achieving promising results that are competitive to state-of-the-art methods. The proposed sparse-to-dense strategy rendered our framework about 7.6 times more efficient than the nearest competitor.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

GeoFusion: Geometric Consistency informed Scene Estimation in Dense Clutter

We propose GeoFusion, a SLAM-based scene estimation method for building an object-level semantic map in dense clutter. In dense clutter, objects are often in close contact and severe occlusions, which brings more false detections and noisy pose estimates from existing perception methods. To solve these problems, our key insight is to consider geometric consistency at the object level within a general SLAM framework. The geometric consistency is defined in two parts: geometric consistency score and geometric relation. The geometric consistency score describes the compatibility between object geometry model and observation point cloud. Meanwhile, it provides a reliable measure to filter out false positives in data association. The geometric relation represents the relationship (e.g. contact) between geometric features (e.g. planes) among objects. The geometric relation makes the graph optimization for poses more robust and accurate. GeoFusion can robustly and efficiently infer the object labels, 6D object poses, and spatial relations from continuous noisy semantic measurements. We quantitatively evaluate our method using observations from a Fetch mobile manipulation robot. Our results demonstrate greater robustness against false estimates than frame-by-frame pose estimation from the state-of-the-art convolutional neural network.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Getting to 99% Accuracy in Interactive Segmentation

Interactive object cutout tools are the cornerstone of the image editing workflow. Recent deep-learning based interactive segmentation algorithms have made significant progress in handling complex images and rough binary selections can typically be obtained with just a few clicks. Yet, deep learning techniques tend to plateau once this rough selection has been reached. In this work, we interpret this plateau as the inability of current algorithms to sufficiently leverage each user interaction and also as the limitations of current training/testing datasets. We propose a novel interactive architecture and a novel training scheme that are both tailored to better exploit the user workflow. We also show that significant improvements can be further gained by introducing a synthetic training dataset that is specifically designed for complex object boundaries. Comprehensive experiments support our approach, and our network achieves state of the art performance.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Incorporating Reinforced Adversarial Learning in Autoregressive Image Generation

Autoregressive models recently achieved comparable results versus state-of-the-art Generative Adversarial Networks (GANs) with the help of Vector Quantized Variational AutoEncoders (VQ-VAE). However, autoregressive models have several limitations such as exposure bias and their training objective does not guarantee visual fidelity. To address these limitations, we propose to use Reinforced Adversarial Learning (RAL) based on policy gradient optimization for autoregressive models. By applying RAL, we enable a similar process for training and testing to address the exposure bias issue. In addition, visual fidelity has been further optimized with adversarial loss inspired by their strong counterparts: GANs. Due to the slow sampling speed of autoregressive models, we propose to use partial generation for faster training. RAL also empowers the collaboration between different modules of the VQ-VAE framework. To our best knowledge, the proposed method is first to enable adversarial learning in autoregressive models for image generation. Experiments on synthetic and real-world datasets show improvements over the MLE trained models. The proposed method improves both negative log-likelihood (NLL) and Fréchet Inception Distance (FID), which indicates improvements in terms of visual quality and diversity. The proposed method achieves state-of-the-art results on Celeba for 64 $\times$ 64 image resolution, showing promise for large scale image generation.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

M2KD: Multi-model and Multi-level Knowledge Distillation for Incremental Learning

Incremental learning targets at achieving good performance on new categories without forgetting old ones. Knowledge distillation has been shown critical in preserving the performance on old classes. Conventional methods, however, sequentially distill knowledge only from the last model, leading to performance degradation on the old classes in later incremental learning steps. In this paper, we propose a multi-model and multi-level knowledge distillation strategy. Instead of sequentially distilling knowledge only from the last model, we directly leverage all previous model snapshots. In addition, we incorporate an auxiliary distillation to further preserve knowledge encoded at the intermediate feature levels. To make the model more memory efficient, we adapt mask based pruning to reconstruct all previous models with a small memory footprint. Experiments on standard incremental learning benchmarks show that our method preserves the knowledge on old classes better and improves the overall performance over standard distillation techniques.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Minimizing FLOPs to Learn Efficient Sparse Representations

Deep representation learning has become one of the most widely adopted approaches for visual search, recommendation, and identification. Retrieval of such representations from a large database is however computationally challenging. Approximate methods based on learning compact representations, have been widely explored for this problem, such as locality sensitive hashing, product quantization, and PCA. In this work, in contrast to learning compact representations, we propose to learn high dimensional and sparse representations that have similar representational capacity as dense embeddings while being more efficient due to sparse matrix multiplication operations which can be much faster than dense multiplication. Following the key insight that the number of operations decreases quadratically with the sparsity of embeddings provided the non-zero entries are distributed uniformly across dimensions, we propose a novel approach to learn such distributed sparse embeddings via the use of a carefully constructed regularization function that directly minimizes a continuous relaxation of the number of floating-point operations (FLOPs) incurred during retrieval. Our experiments show that our approach is competitive to the other baselines and yields a similar or better speed-vs-accuracy tradeoff on practical datasets.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Moiré effects in graphene--hBN heterostructures

Encapsulating graphene in hexagonal Boron Nitride has several advantages: the highest mobilities reported to date are achieved in this way, and precise nanostructuring of graphene becomes feasible through the protective hBN layers. Nevertheless, subtle effects may arise due to the differing lattice constants of graphene and hBN, and due to the twist angle between the graphene and hBN lattices. Here, we use a recently developed model which allows us to perform band structure and magnetotransport calculations of such structures, and show that with a proper account of the moiré physics an excellent agreement with experiments can be achieved, even for complicated structures such as disordered graphene, or antidot lattices on a monolayer hBN with a relative twist angle. Calculations of this kind are essential to a quantitative modeling of twistronic devices.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Multiple Sound Sources Localization from Coarse to Fine

How to visually localize multiple sound sources in unconstrained videos is a formidable problem, especially when lack of the pairwise sound-object annotations. To solve this problem, we develop a two-stage audiovisual learning framework that disentangles audio and visual representations of different categories from complex scenes, then performs cross-modal feature alignment in a coarse-to-fine manner. Our model achieves state-of-the-art results on public dataset of localization, as well as considerable performance on multi-source sound localization in complex scenes. We then employ the localization results for sound separation and obtain comparable performance to existing methods. These outcomes demonstrate our model&#39;s ability in effectively aligning sounds with specific visual sources. Code is available at https://github.com/shvdiwnkozbw/Multi-Source-Sound-Localization

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Rademacher upper bounds for cross-validation errors with an application to the lasso

We establish a general upper bound for $K$-fold cross-validation ($K$-CV) errors that can be adapted to many $K$-CV-based estimators and learning algorithms. Based on Rademacher complexity of the model and the Orlicz-$Ψ_ν$ norm of the error process, the CV error upper bound applies to both light-tail and heavy-tail error distributions. We also extend the CV error upper bound to $β$-mixing data using the technique of independent blocking. We provide a Python package (\texttt{CVbound}, \url{https://github.com/isaac2math}) for computing the CV error upper bound in $K$-CV-based algorithms. Using the lasso as an example, we demonstrate in simulations that the upper bounds are tight and stable across different parameter settings and random seeds. As well as accurately bounding the CV errors for the lasso, the minimizer of the new upper bounds can be used as a criterion for variable selection. Compared with the CV-error minimizer, simulations show that tuning the lasso penalty parameter according to the minimizer of the upper bound yields a more sparse and more stable model that retains all of the relevant variables.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Video Question Answering on Screencast Tutorials

This paper presents a new video question answering task on screencast tutorials. We introduce a dataset including question, answer and context triples from the tutorial videos for a software. Unlike other video question answering works, all the answers in our dataset are grounded to the domain knowledge base. An one-shot recognition algorithm is designed to extract the visual cues, which helps enhance the performance of video question answering. We also propose several baseline neural network architectures based on various aspects of video contexts from the dataset. The experimental results demonstrate that our proposed models significantly improve the question answering performances by incorporating multi-modal contexts and domain knowledge.

0 citations0 reviewsNo code linkedOpen access