BZPEERReading, trust and collaboration for research
Menu

Discover

GraphFollow connections around a paper, topic or saved view.

Workspaces

HomePick up where your research flow left off.InboxHandle requests, replies and notifications from one place.CollectionsCurate reading lists, evidence sets and shareable dossiers.ResearchSee your private provenance graph, trail and imports.CollaborationMove from discovery into focused discussion rooms.PublishDraft, preview and publish full-text research articles.

Network

ExpertsFind specialists worth following or asking for help.CommunitiesJoin research circles organized around shared work.PartnersSee external organizations active around the same topics.

Opportunities

ListingsBrowse roles, calls and collaborations with clearer fit signals.

Account

Log inReturn to your reading and collaboration workspace.Create accountStart saving work, following people and joining teams.
Log inCreate account

Researcher profile

Kang Zhao

Kang Zhao contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate
Computer VisionComputation and Languagecond-mat.supr-conMachine LearningArtificial Intelligenceastro-ph.IMcond-mat.mtrl-scics.CYhep-exInformation Retrievalmath.NANumerical Analysis

Trust snapshot

Quick read

Trust 21 - EmergingVerification L1Unclaimed author
14works
0followers
12topics
4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Identity and collaboration

How to connect with this researcher

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Log in to claim

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Research graph

See the researcher in context

Open full explorer

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

14 published item(s)

preprint2023arXiv

Semi-MAE: Masked Autoencoders for Semi-supervised Vision Transformers

Vision Transformer (ViT) suffers from data scarcity in semi-supervised learning (SSL). To alleviate this issue, inspired by masked autoencoder (MAE), which is a data-efficient self-supervised learner, we propose Semi-MAE, a pure ViT-based SSL framework consisting of a parallel MAE branch to assist the visual representation learning and make the pseudo labels more accurate. The MAE branch is designed as an asymmetric architecture consisting of a lightweight decoder and a shared-weights encoder. We feed the weakly-augmented unlabeled data with a high masking ratio to the MAE branch and reconstruct the missing pixels. Semi-MAE achieves 75.9% top-1 accuracy on ImageNet with 10% labels, surpassing prior state-of-the-art in semi-supervised image classification. In addition, extensive experiments demonstrate that Semi-MAE can be readily used for other ViT models and masked image modeling methods.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

CATCH: Chasing All Transients Constellation Hunters Space Mission

In time-domain astronomy, a substantial number of transients will be discovered by multi-wavelength and multi-messenger observatories, posing a great challenge for follow-up capabilities. We have thus proposed an intelligent X-ray constellation, the Chasing All Transients Constellation Hunters (CATCH) space mission. Consisting of 126 micro-satellites in three types, CATCH will have the capability to perform follow-up observations for a large number of different types of transients simultaneously. Each satellite in the constellation will carry lightweight X-ray optics and use a deployable mast to increase the focal length. The combination of different optics and detector systems enables different types of satellites to have multiform observation capabilities, including timing, spectroscopy, imaging, and polarization. Controlled by the intelligent system, different satellites can cooperate to perform uninterrupted monitoring, all-sky follow-up observations, and scanning observations with a flexible field of view (FOV) and multi-dimensional observations. Therefore, CATCH will be a powerful mission to study the dynamic universe. Here, we present the current design of the spacecraft, optics, detector system, constellation configuration and observing modes, as well as the development plan.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Consistent Representation Learning for Continual Relation Extraction

Continual relation extraction (CRE) aims to continuously train a model on data with new relations while avoiding forgetting old ones. Some previous work has proved that storing a few typical samples of old relations and replaying them when learning new relations can effectively avoid forgetting. However, these memory-based methods tend to overfit the memory samples and perform poorly on imbalanced datasets. To solve these challenges, a consistent representation learning method is proposed, which maintains the stability of the relation embedding by adopting contrastive learning and knowledge distillation when replaying memory. Specifically, supervised contrastive learning based on a memory bank is first used to train each new task so that the model can effectively learn the relation representation. Then, contrastive replay is conducted of the samples in memory and makes the model retain the knowledge of historical relations through memory knowledge distillation to prevent the catastrophic forgetting of the old task. The proposed method can better learn consistent representations to alleviate forgetting effectively. Extensive experiments on FewRel and TACRED datasets show that our method significantly outperforms state-of-the-art baselines and yield strong robustness on the imbalanced dataset.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Strong-Coupling Superconductivity with $T_c$ $\sim$ 10.8 K Induced by P Doping in the Topological Semimetal Mo$_5$Si$_3$

By performing P doping on the Si sites in the topological semimetal Mo$_5$Si$_3$, we discover strong-coupling superconductivity in Mo$_5$Si$_{3-x}$P$_x$ (0.5 $\le$ $x$ $\le$ 2.0). Mo$_5$Si$_3$ crystallizes in the W$_5$Si$_3$-type structure with space group of $I4/mcm$ (No. 140), and is not a superconductor itself. Upon P doping, the lattice parameter $a$ decreases while $c$ increases monotonously. Bulk superconductivity is revealed in Mo$_5$Si$_{3-x}$P$_x$ (0.5 $\le$ $x$ $\le$ 2.0) from resistivity, magnetization, and heat capacity measurements. $T_c$ in Mo$_5$Si$_{1.5}$P$_{1.5}$ reaches as high as 10.8 K, setting a new record among the W$_5$Si$_3$-type superconductors. The upper and lower critical fields for Mo$_5$Si$_{1.5}$P$_{1.5}$ are 14.56 T and 105 mT, respectively. Moreover, Mo$_5$Si$_{1.5}$P$_{1.5}$ is found to be a fully gapped superconductor with strong electron-phonon coupling. First-principles calculations suggest that the enhancement of electron-phonon coupling is possibly due to the shift of the Fermi level, which is induced by electron doping. The calculations also reveal the nontrivial band topology in Mo$_5$Si$_3$. The $T_c$ and upper critical field in Mo$_5$Si$_{3-x}$P$_x$ are fairly high among pseudobinary compounds. Both of them are higher than those in NbTi, making future applications promising. Our results suggest that the W$_5$Si$_3$-type compounds are ideal platforms to search for new superconductors. By examinations of their band topologies, more candidates for topological superconductors can be expected in this structural family.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Distribution Adaptive INT8 Quantization for Training CNNs

Researches have demonstrated that low bit-width (e.g., INT8) quantization can be employed to accelerate the inference process. It makes the gradient quantization very promising since the backward propagation requires approximately twice more computation than forward one. Due to the variability and uncertainty of gradient distribution, a lot of methods have been proposed to attain training stability. However, most of them ignore the channel-wise gradient distributions and the impact of gradients with different magnitudes, resulting in the degradation of final accuracy. In this paper, we propose a novel INT8 quantization training framework for convolutional neural network to address the above issues. Specifically, we adopt Gradient Vectorized Quantization to quantize the gradient, based on the observation that layer-wise gradients contain multiple distributions along the channel dimension. Then, Magnitude-aware Clipping Strategy is introduced by taking the magnitudes of gradients into consideration when minimizing the quantization error, and we present a theoretical derivation to solve the quantization parameters of different distributions. Experimental results on broad range of computer vision tasks, such as image classification, object detection and video classification, demonstrate that the proposed Distribution Adaptive INT8 Quantization training method has achieved almost lossless training accuracy for different backbones, including ResNet, MobileNetV2, InceptionV3, VGG and AlexNet, which is superior to the state-of-the-art techniques. Moreover, we further implement the INT8 kernel that can accelerate the training iteration more than 200% under the latest Turing architecture, i.e., our method excels on both training accuracy and speed.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Large-Scale Training System for 100-Million Classification at Alibaba

In the last decades, extreme classification has become an essential topic for deep learning. It has achieved great success in many areas, especially in computer vision and natural language processing (NLP). However, it is very challenging to train a deep model with millions of classes due to the memory and computation explosion in the last output layer. In this paper, we propose a large-scale training system to address these challenges. First, we build a hybrid parallel training framework to make the training process feasible. Second, we propose a novel softmax variation named KNN softmax, which reduces both the GPU memory consumption and computation costs and improves the throughput of training. Then, to eliminate the communication overhead, we propose a new overlapping pipeline and a gradient sparsification method. Furthermore, we design a fast continuous convergence strategy to reduce total training iterations by adaptively adjusting learning rate and updating model parameters. With the help of all the proposed methods, we gain 3.9$\times$ throughput of our training system and reduce almost 60\% of training iterations. The experimental results show that using an in-house 256 GPUs cluster, we could train a classifier of 100 million classes on Alibaba Retail Product Dataset in about five days while achieving a comparable accuracy with the naive softmax training process.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Large-Scale Visual Search with Binary Distributed Graph at Alibaba

Graph-based approximate nearest neighbor search has attracted more and more attentions due to its online search advantages. Numbers of methods studying the enhancement of speed and recall have been put forward. However, few of them focus on the efficiency and scale of offline graph-construction. For a deployed visual search system with several billions of online images in total, building a billion-scale offline graph in hours is essential, which is almost unachievable by most existing methods. In this paper, we propose a novel algorithm called Binary Distributed Graph to solve this problem. Specifically, we combine binary codes with graph structure to speedup online and offline procedures, and achieve comparable performance with the ones in real-value based scenarios by recalling more binary candidates. Furthermore, the graph-construction is optimized to completely distributed implementation, which significantly accelerates the offline process and gets rid of the limitation of memory and disk within a single machine. Experimental comparisons on Alibaba Commodity Data Set (more than three billion images) show that the proposed method outperforms the state-of-the-art with respect to the online/offline trade-off.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

TEXTOIR: An Integrated and Visualized Platform for Text Open Intent Recognition

TEXTOIR is the first integrated and visualized platform for text open intent recognition. It is composed of two main modules: open intent detection and open intent discovery. Each module integrates most of the state-of-the-art algorithms and benchmark intent datasets. It also contains an overall framework connecting the two modules in a pipeline scheme. In addition, this platform has visualized tools for data and model management, training, evaluation and analysis of the performance from different aspects. TEXTOIR provides useful toolkits and convenient visualized interfaces for each sub-module (Toolkit code: https://github.com/thuiar/TEXTOIR), and designs a framework to implement a complete process to both identify known intents and discover open intents (Demo code: https://github.com/thuiar/TEXTOIR-DEMO).

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Virtual ID Discovery from E-commerce Media at Alibaba: Exploiting Richness of User Click Behavior for Visual Search Relevance

Visual search plays an essential role for E-commerce. To meet the search demands of users and promote shopping experience at Alibaba, visual search relevance of real-shot images is becoming the bottleneck. Traditional visual search paradigm is usually based upon supervised learning with labeled data. However, large-scale categorical labels are required with expensive human annotations, which limits its applicability and also usually fails in distinguishing the real-shot images. In this paper, we propose to discover Virtual ID from user click behavior to improve visual search relevance at Alibaba. As a totally click-data driven approach, we collect various types of click data for training deep networks without any human annotations at all. In particular, Virtual ID are learned as classification supervision with co-click embedding, which explores image relationship from user co-click behaviors to guide category prediction and feature learning. Concretely, we deploy Virtual ID Category Network by integrating first-clicks and switch-clicks as regularizer. Incorporating triplets and list constraints, Virtual ID Feature Network is trained in a joint classification and ranking manner. Benefiting from exploration of user click data, our networks are more effective to encode richer supervision and better distinguish real-shot images in terms of category and feature. To validate our method for visual search relevance, we conduct an extensive set of offline and online experiments on the collected real-shot images. We consistently achieve better experimental results across all components, compared with alternative and state-of-the-art methods.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Visual Search at Alibaba

This paper introduces the large scale visual search algorithm and system infrastructure at Alibaba. The following challenges are discussed under the E-commercial circumstance at Alibaba (a) how to handle heterogeneous image data and bridge the gap between real-shot images from user query and the online images. (b) how to deal with large scale indexing for massive updating data. (c) how to train deep models for effective feature representation without huge human annotations. (d) how to improve the user engagement by considering the quality of the content. We take advantage of large image collection of Alibaba and state-of-the-art deep learning techniques to perform visual search at scale. We present solutions and implementation details to overcome those problems and also share our learnings from building such a large scale commercial visual search engine. Specifically, model and search-based fusion approach is introduced to effectively predict categories. Also, we propose a deep CNN model for joint detection and feature learning by mining user click behavior. The binary index engine is designed to scale up indexing without compromising recall and precision. Finally, we apply all the stages into an end-to-end system architecture, which can simultaneously achieve highly efficient and scalable performance adapting to real-shot images. Extensive experiments demonstrate the advancement of each module in our system. We hope visual search at Alibaba becomes more widely incorporated into today's commercial applications.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Boundary Regularized Building Footprint Extraction From Satellite Images Using Deep Neural Network

In recent years, an ever-increasing number of remote satellites are orbiting the Earth which streams vast amount of visual data to support a wide range of civil, public and military applications. One of the key information obtained from satellite imagery is to produce and update spatial maps of built environment due to its wide coverage with high resolution data. However, reconstructing spatial maps from satellite imagery is not a trivial vision task as it requires reconstructing a scene or object with high-level representation such as primitives. For the last decade, significant advancement in object detection and representation using visual data has been achieved, but the primitive-based object representation still remains as a challenging vision task. Thus, a high-quality spatial map is mainly produced through complex labour-intensive processes. In this paper, we propose a novel deep neural network, which enables to jointly detect building instance and regularize noisy building boundary shapes from a single satellite imagery. The proposed deep learning method consists of a two-stage object detection network to produce region of interest (RoI) features and a building boundary extraction network using graph models to learn geometric information of the polygon shapes. Extensive experiments show that our model can accomplish multi-tasks of object localization, recognition, semantic labelling and geometric shape extraction simultaneously. In terms of building extraction accuracy, computation efficiency and boundary regularization performance, our model outperforms the state-of-the-art baseline models.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Eigenvalue embedding of undamped piezoelectric structure system with no-spillover

In this paper, the eigenvalue embedding problem of the undamped piezoelectric structure system with no-spillover (EEP-PS) is considered. It aims to update the original system to a new undamped piezoelectric structure system, such that some eigenpairs are replaced by newly given or measured ones, while the remaining eigenpairs are kept unchanged. A set of parametric solutions to the EEP-PS are provided and the performance of the proposed algorithms are illustrated by numerical examples.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

How Search Engine Advertising Affects Sales over Time: An Empirical Investigation

As a mainstream marketing channel on the Internet, Search Engine Advertising (SEA) has a huge business impact and attracts a plethora of attention from both academia and industry. One important goal of advertising is to increase sales. Nevertheless, while previous research has studied multiple factors that are potentially related to the outcome of SEA campaigns, effects of these factors on actual sales generated by SEA remain understudied. It is also unclear whether and how such effects change over time in highly dynamic SEA campaigns. As the first empirical investigation of the dynamic advertisement-sales relationship in SEA, this study builds an advertising response model within a time-varying coefficient (TVC) modeling framework, and estimates the model using a unique dataset from a large E-Commerce retailer in the United States. Results reveal the effects of the advertising expenditure, consumer behaviors and advertisement characteristics on realized sales, and demonstrate that such effects on sales do change over time in non-linear ways. More importantly, we find that carryover has a stronger effect in generating sales than direct response does, conversion rate is much more important than click-through rate, and ad position does not have significant effects on sales. These findings have direct implications for advertisers to launch more effective SEA campaigns.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Reemergence of superconductivity in pressurized quasi-one-dimensional superconductor K2Mo3As3

Here we report a pressure-induced reemergence of superconductivity in recently discovered superconductor K2Mo3As3, which is the first experimental case observed in quasi-one-dimensional superconductors. We find that, after full suppression of the ambient-pressure superconducting (SC-I) state at 8.7 GPa, an intermediary non-superconducting state sets in and prevails to the pressure up to 18.2 GPa, however, above this pressure a new superconducting (SC-II) state appears unexpectedly. High pressure x-ray diffraction measurements demonstrate that the pressure-induced dramatic change of the lattice parameter c contributes mainly to the emergence of the SC-II state. Combined with the theioretical calculations on band strcture, our results suggest that the reemergemce of superconductivity is associated with the change of the complicated interplay among different orbital electrons, driven by the pressure-induced unisotropic change of the lattice.

0 citations0 reviewsNo code linkedOpen access