Trust snapshot

Quick read

Trust 21 - EmergingVerification L1Unclaimed author
31works
0followers
19topics
4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Identity and collaboration

How to connect with this researcher

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Log in to claim

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Research graph

See the researcher in context

Open full explorer

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

31 published item(s)

preprint2026arXiv

Beyond Heuristics: Learnable Density Control for 3D Gaussian Splatting

While 3D Gaussian Splatting (3DGS) has demonstrated impressive real-time rendering performance, its efficacy remains constrained by a reliance on heuristic density control. Despite numerous refinements to these handcrafted rules, such methods inherently lack the flexibility to adapt to diverse scenes with complex geometries. In this paper, we propose a paradigm shift for density control from rigid heuristics to fully learnable policies. Specifically, we introduce \textbf{LeGS}, a framework that reformulates density control as a parameterized policy network optimized via Reinforcement Learning (RL). Central to our approach is the tailored effective reward function grounded in sensitivity analysis, which precisely quantifies the marginal contribution of individual Gaussians to reconstruction quality. To maintain computational tractability, we derive a closed-form solution that reduces the complexity of reward calculation from $O(N^2)$ to $O(N)$. Extensive experiments on the Mip-NeRF 360, Tanks \& Temples, and Deep Blending datasets demonstrate that \textbf{LeGS} significantly outperforms state-of-the-art methods, striking a superior balance between reconstruction quality and efficiency. The code will be released at https://github.com/AaronNZH/LeGS

preprint2024arXiv

BCLNet: Bilateral Consensus Learning for Two-View Correspondence Pruning

Correspondence pruning aims to establish reliable correspondences between two related images and recover relative camera motion. Existing approaches often employ a progressive strategy to handle the local and global contexts, with a prominent emphasis on transitioning from local to global, resulting in the neglect of interactions between different contexts. To tackle this issue, we propose a parallel context learning strategy that involves acquiring bilateral consensus for the two-view correspondence pruning task. In our approach, we design a distinctive self-attention block to capture global context and parallel process it with the established local context learning module, which enables us to simultaneously capture both local and global consensuses. By combining these local and global consensuses, we derive the required bilateral consensus. We also design a recalibration block, reducing the influence of erroneous consensus information and enhancing the robustness of the model. The culmination of our efforts is the Bilateral Consensus Learning Network (BCLNet), which efficiently estimates camera pose and identifies inliers (true correspondences). Extensive experiments results demonstrate that our network not only surpasses state-of-the-art methods on benchmark datasets but also showcases robust generalization abilities across various feature extraction techniques. Noteworthily, BCLNet obtains 3.98\% mAP5$^{\circ}$ gains over the second best method on unknown outdoor dataset, and obviously accelerates model training speed. The source code will be available at: https://github.com/guobaoxiao/BCLNet.

preprint2023arXiv

A Two-stage Fine-tuning Strategy for Generalizable Manipulation Skill of Embodied AI

The advent of Chat-GPT has led to a surge of interest in Embodied AI. However, many existing Embodied AI models heavily rely on massive interactions with training environments, which may not be practical in real-world situations. To this end, the Maniskill2 has introduced a full-physics simulation benchmark for manipulating various 3D objects. This benchmark enables agents to be trained using diverse datasets of demonstrations and evaluates their ability to generalize to unseen scenarios in testing environments. In this paper, we propose a novel two-stage fine-tuning strategy that aims to further enhance the generalization capability of our model based on the Maniskill2 benchmark. Through extensive experiments, we demonstrate the effectiveness of our approach by achieving the 1st prize in all three tracks of the ManiSkill2 Challenge. Our findings highlight the potential of our method to improve the generalization abilities of Embodied AI models and pave the way for their ractical applications in real-world scenarios. All codes and models of our solution is available at https://github.com/xtli12/GXU-LIPE.git

preprint2023arXiv

Efficient Approximation of Gromov-Wasserstein Distance Using Importance Sparsification

As a valid metric of metric-measure spaces, Gromov-Wasserstein (GW) distance has shown the potential for matching problems of structured data like point clouds and graphs. However, its application in practice is limited due to the high computational complexity. To overcome this challenge, we propose a novel importance sparsification method, called \textsc{Spar-GW}, to approximate GW distance efficiently. In particular, instead of considering a dense coupling matrix, our method leverages a simple but effective sampling strategy to construct a sparse coupling matrix and update it with few computations. The proposed \textsc{Spar-GW} method is applicable to the GW distance with arbitrary ground cost, and it reduces the complexity from $O(n^4)$ to $O(n^{2+δ})$ for an arbitrary small $δ>0$. Theoretically, the convergence and consistency of the proposed estimation for GW distance are established under mild regularity conditions. In addition, this method can be extended to approximate the variants of GW distance, including the entropic GW distance, the fused GW distance, and the unbalanced GW distance. Experiments show the superiority of our \textsc{Spar-GW} to state-of-the-art methods in both synthetic and real-world tasks.

preprint2023arXiv

Multiplayer Battle Game-Inspired Optimizer for Complex Optimization Problems

Various popular multiplayer battle royale games share a lot of common elements. Drawing from our observations, we summarized these shared characteristics and subsequently proposed a novel heuristic algorithm named multiplayer battle game-inspired optimizer (MBGO). The proposed MBGO streamlines mainstream multiplayer battle royale games into two discrete phases: movement and battle. Specifically, the movement phase incorporates the principles of commonly encountered ``safe zones'' to incentivize participants to relocate to areas with a higher survival potential. The battle phase simulates a range of strategies adopted by players in various situations to enhance the diversity of the population. To evaluate and analyze the performance of the proposed MBGO, we executed it alongside eight other algorithms, including three classics and five latest ones, across multiple diverse dimensions within the CEC2017 and CEC2020 benchmark functions. In addition, we employed several industrial design problems to evaluate the scalability and practicality of the proposed MBGO. The results of the statistical analysis reveal that the novel MBGO demonstrates significant competitiveness, excelling not only in convergence speed, but also in achieving high levels of convergence accuracy across both benchmark functions and real-world problems.

preprint2022arXiv

A strong multiplicity one theorem for dimension data

We call the dimension data $\mathscr{D}_{H_{1}}$ and $\mathscr{D}_{H_{2}}$ of two closed subgroups $H_{1}$ and $H_{2}$ of a given compact Lie group $G$ {\it almost equal} if $\mathscr{D}_{H_{1}}(ρ)=\mathscr{D}_{H_{2}}(ρ)$ for all but finitely many irreducible complex linear representations $ρ$ of $G$ up to equivalence. When $G$ is connected, we show that: if $\mathscr{D}_{H_{1}}$ and $\mathscr{D}_{H_{2}}$ are almost equal, then they are equal. When $G$ is non-connected, $G^{0}\subset H_{1}\cap H_{2}$ is a trivial sufficient condition for $\mathscr{D}_{H_{1}}$ and $\mathscr{D}_{H_{2}}$ to be almost equal. In this case assume that $\mathscr{D}_{H_{1}}$ and $\mathscr{D}_{H_{2}}$ are almost equal but non-equal. We show strong relations between $H_{1}$ and $H_{2}$ and we construct an example which indicates that $G^{0}\subset H_{1}\cap H_{2}$ is not a necessary condition.

preprint2022arXiv

An optimal transport approach for selecting a representative subsample with application in efficient kernel density estimation

Subsampling methods aim to select a subsample as a surrogate for the observed sample. Such methods have been used pervasively in large-scale data analytics, active learning, and privacy-preserving analysis in recent decades. Instead of model-based methods, in this paper, we study model-free subsampling methods, which aim to identify a subsample that is not confined by model assumptions. Existing model-free subsampling methods are usually built upon clustering techniques or kernel tricks. Most of these methods suffer from either a large computational burden or a theoretical weakness. In particular, the theoretical weakness is that the empirical distribution of the selected subsample may not necessarily converge to the population distribution. Such computational and theoretical limitations hinder the broad applicability of model-free subsampling methods in practice. We propose a novel model-free subsampling method by utilizing optimal transport techniques. Moreover, we develop an efficient subsampling algorithm that is adaptive to the unknown probability density function. Theoretically, we show the selected subsample can be used for efficient density estimation by deriving the convergence rate for the proposed subsample kernel density estimator. We also provide the optimal bandwidth for the proposed estimator. Numerical studies on synthetic and real-world datasets demonstrate the performance of the proposed method is superior.

preprint2022arXiv

Do We Need to Penalize Variance of Losses for Learning with Label Noise?

Algorithms which minimize the averaged loss have been widely designed for dealing with noisy labels. Intuitively, when there is a finite training sample, penalizing the variance of losses will improve the stability and generalization of the algorithms. Interestingly, we found that the variance should be increased for the problem of learning with noisy labels. Specifically, increasing the variance will boost the memorization effects and reduce the harmfulness of incorrect labels. By exploiting the label noise transition matrix, regularizers can be easily designed to reduce the variance of losses and be plugged in many existing algorithms. Empirically, the proposed method by increasing the variance of losses significantly improves the generalization ability of baselines on both synthetic and real-world datasets.

preprint2022arXiv

High-Performance Transmission Mechanism Design of Multi-Stream Carrier Aggregation for 5G Non-Standalone Network

Multi-stream carrier aggregation is a key technology to expand bandwidth and improve the throughput of the fifth-generation wireless communication systems. However, due to the diversified propagation properties of different frequency bands, the traffic migration task is much more challenging, especially in hybrid sub-6 GHz and millimeter wave bands scenario. Existing schemes either neglected to consider the transmission rate difference between multi-stream carrier, or only consider simple low mobility scenario. In this paper, we propose a low-complexity traffic splitting algorithm based on fuzzy proportional integral derivative control mechanism. The proposed algorithm only relies on the local radio link control buffer information of sub-6 GHz and mmWave bands, while frequent feedback from user equipment (UE) side is minimized. As shown in the numerical examples, the proposed traffic splitting mechanism can achieve more than 90% link resource utilization ratio for different UE transmission requirements with different mobilities, which corresponds to 10% improvement if compared with conventional baselines.

preprint2022arXiv

Hyper-relationship Learning Network for Scene Graph Generation

Generating informative scene graphs from images requires integrating and reasoning from various graph components, i.e., objects and relationships. However, current scene graph generation (SGG) methods, including the unbiased SGG methods, still struggle to predict informative relationships due to the lack of 1) high-level inference such as transitive inference between relationships and 2) efficient mechanisms that can incorporate all interactions of graph components. To address the issues mentioned above, we devise a hyper-relationship learning network, termed HLN, for SGG. Specifically, the proposed HLN stems from hypergraphs and two graph attention networks (GATs) are designed to infer relationships: 1) the object-relationship GAT or OR-GAT to explore interactions between objects and relationships, and 2) the hyper-relationship GAT or HR-GAT to integrate transitive inference of hyper-relationships, i.e., the sequential relationships between three objects for transitive reasoning. As a result, HLN significantly improves the performance of scene graph generation by integrating and reasoning from object interactions, relationship interactions, and transitive inference of hyper-relationships. We evaluate HLN on the most popular SGG dataset, i.e., the Visual Genome dataset, and the experimental results demonstrate its great superiority over recent state-of-the-art methods. For example, the proposed HLN improves the recall per relationship from 11.3\% to 13.1\%, and maintains the recall per image from 19.8\% to 34.9\%. We will release the source code and pretrained models on GitHub.

preprint2022arXiv

Learning Disentangled Representations for Controllable Human Motion Prediction

Generative model-based motion prediction techniques have recently realized predicting controlled human motions, such as predicting multiple upper human body motions with similar lower-body motions. However, to achieve this, the state-of-the-art methods require either subsequently learning mapping functions to seek similar motions or training the model repetitively to enable control over the desired portion of body. In this paper, we propose a novel framework to learn disentangled representations for controllable human motion prediction. Our network involves a conditional variational auto-encoder (CVAE) architecture to model full-body human motion, and an extra CVAE path to learn only the corresponding partial-body (e.g., lower-body) motion. Specifically, the inductive bias imposed by the extra CVAE path encourages two latent variables in two paths to respectively govern separate representations for each partial-body motion. With a single training, our model is able to provide two types of controls for the generated human motions: (i) strictly controlling one portion of human body and (ii) adaptively controlling the other portion, by sampling from a pair of latent spaces. Additionally, we extend and adapt a sampling strategy to our trained model to diversify the controllable predictions. Our framework also potentially allows new forms of control by flexibly customizing the input for the extra CVAE path. Extensive experimental results and ablation studies demonstrate that our approach is capable of predicting state-of-the-art controllable human motions both qualitatively and quantitatively.

preprint2022arXiv

Multi-model Ensemble Learning Method for Human Expression Recognition

Analysis of human affect plays a vital role in human-computer interaction (HCI) systems. Due to the difficulty in capturing large amounts of real-life data, most of the current methods have mainly focused on controlled environments, which limit their application scenarios. To tackle this problem, we propose our solution based on the ensemble learning method. Specifically, we formulate the problem as a classification task, and then train several expression classification models with different types of backbones--ResNet, EfficientNet and InceptionNet. After that, the outputs of several models are fused via model ensemble method to predict the final results. Moreover, we introduce the multi-fold ensemble method to train and ensemble several models with the same architecture but different data distributions to enhance the performance of our solution. We conduct many experiments on the AffWild2 dataset of the ABAW2022 Challenge, and the results demonstrate the effectiveness of our solution.

preprint2022arXiv

Statistical learning for train delays and influence of winter climate and atmospheric icing

This study investigated the climate effect under consecutive winters on the arrival delay of high-speed passenger trains in northern Sweden. Novel statistical learning approaches, including inhomogeneous Markov chain model and stratified Cox model, were adopted to account for the time-varying risks of train delays. The inhomogeneous Markov chain modelling for the arrival delays has used several covariates, including weather variables, train operational direction, and findings from the primary delay analysis through stratified Cox model. The results showed that the weather variables, such as temperature, snow depth, ice/snow precipitation, and train operational direction, significantly impact the arrival delay. The performance of the fitted inhomogeneous Markov chain model was evaluated by the walk-forward validation method. The averaged mean absolute errors between the expected rates and the observed rates of the arrival delay over the train line was obtained at the level of 0.088, which implies that approximately 9% of trains may be misclassified as having arrival delays by the fitted model at a measuring point on the train line.

preprint2022arXiv

The functorial source problem via dimension data

For an automorphic representation $π$ of Ramanujan type, there is a conjectural subgroup $\mathcal{H}_π$ of the Langlands L-group $^{L}G$ associated to $π$, called the {\it functional source} of $π$. The functorial source problem proposed by Langlands and refined by Arthur intends to determine $\mathcal{H}_π$ via analytic and arithmetic data of $π$. In this paper, we consider the functorial source problem of automorphic representations of a split group, a unitary group, or an orthogonal group which do not come from endoscopy and have minimal possible ramification. In this setting, $\mathcal{H}_π$ must be an S-subgroup of $^{L}G$. We approach the functorial source problem by proving distinction and linear independence among dimension data of S-subgroups. Nice results along this direction are shown in this paper. We define a notion of quasi root system and use it as the key tool for studying S-subgroups and their dimension data.

preprint2022arXiv

Understanding Robust Overfitting of Adversarial Training and Beyond

Robust overfitting widely exists in adversarial training of deep networks. The exact underlying reasons for this are still not completely understood. Here, we explore the causes of robust overfitting by comparing the data distribution of \emph{non-overfit} (weak adversary) and \emph{overfitted} (strong adversary) adversarial training, and observe that the distribution of the adversarial data generated by weak adversary mainly contain small-loss data. However, the adversarial data generated by strong adversary is more diversely distributed on the large-loss data and the small-loss data. Given these observations, we further designed data ablation adversarial training and identify that some small-loss data which are not worthy of the adversary strength cause robust overfitting in the strong adversary mode. To relieve this issue, we propose \emph{minimum loss constrained adversarial training} (MLCAT): in a minibatch, we learn large-loss data as usual, and adopt additional measures to increase the loss of the small-loss data. Technically, MLCAT hinders data fitting when they become easy to learn to prevent robust overfitting; philosophically, MLCAT reflects the spirit of turning waste into treasure and making the best use of each adversarial data; algorithmically, we designed two realizations of MLCAT, and extensive experiments demonstrate that MLCAT can eliminate robust overfitting and further boost adversarial robustness.

preprint2021arXiv

Existence of weak solutions to a generalized nonlinear multi-layered fluid-structure interaction problem with the Navier-slip boundary conditions

We consider a fluid-structure interaction problem with Navier-slip boundary conditions in which the fluid is considered as a non-Newtonian fluid and the structure is described by a nonlinear multi-layered model. The fluid domain is driven by a nonlinear elastic shell and thus is not fixed. To simplify the problem, we map the moving fluid domain into a fixed domain by applying an arbitrary Lagrange Euler mapping. Unlike the classical method by which we can consider the problem as its entirety, we utilize the time-discretization and split the problem into a fluid subproblem and a structure subproblem by an operator splitting scheme. Since the structure subproblem is nonlinear, Lax-Milgram lemma does not hold. Here we prove the existence and uniqueness by means of the traditional semigroup theory. Noticing that the Non-Newtonian fluid possesses a $ p- $Laplacian structure, we show the existence and uniqueness of solutions to the fluid subproblem by considering the Browder-Minty theorem. With the uniform energy estimates, we deduce the weak and weak* convergence respectively. By a generalized Aubin-Lions-Simon Lemma proposed by Muha and Canić [J. Differential Equations {\bf 266} (2019), 8370--8418], we obtain the strong convergence. Finally, we construct the test functions and pass the approximate weak formulation to the limit as time step goes to zero with the convergence results.

preprint2021arXiv

QoS-Driven Resource Optimization for Intelligent Fog Radio Access Network: A Dynamic Power Allocation Perspective

The fog radio access network (Fog-RAN) has been considered a promising wireless access architecture to help shorten the communication delay and relieve the large data delivery burden over the backhaul links. However, limited by conventional inflexible communication design, Fog-RAN cannot be used in some complex communication scenarios. In this study, we focus on investigating a more intelligent Fog-RAN to assist the communication in a high-speed railway environment. Due to the train's continuously moving, the communication should be designed intelligently to adapt to channel variation. Specifically, we dynamically optimize the power allocation in the remote radio heads (RRHs) to minimize the total network power cost considering multiple quality-of-service (QoS) requirements and channel variation. The impact of caching on the power allocation is considered. The dynamic power optimization is analyzed to obtain a closed-form solution in certain cases. The inherent tradeoff among the total network cost, delay and delivery content size is further discussed. To evaluate the performance of the proposed dynamic power allocation, we present an invariant power allocation counterpart as a performance comparison benchmark. The result of our simulation reveals that dynamic power allocation can significantly outperform the invariant power allocation scheme, especially with a random caching strategy or limited caching resources at the RRHs.

preprint2021arXiv

Some new results on dimension datum

In this paper we show three new results concerning dimension datum. Firstly, for two subgroups $H_{1}$($\cong U(2n+1)$) and $H_{2}$($\cong Sp(n)\times SO(2n+2)$) of $SU(4n+2)$, we find a family of pairs of irreducible representations $(τ_1,τ_2)\in\hat{H_{1}}\times\hat{H_{2}}$ such that $\mathscr{D}_{H_1,τ_1}=\mathscr{D}_{H_2,τ_2}$. With this we construct examples of isospectral hermitian vector bundles. Secondly, we show that: $τ$-dimension data of one-dimensional representations of a connected compact Lie group $H$ determine the image of homomorphism from $H$ to a given compact Lie group $G$. Lastly, we improve a compactness result for an isospectral set of normal homogeneous spaces $(G/H,m)$ by allowing the Riemannian metric $m$ vary, but posing a constraint that $G$ is semisimple.

preprint2021arXiv

Sufficient dimension reduction for classification using principal optimal transport direction

Sufficient dimension reduction is used pervasively as a supervised dimension reduction approach. Most existing sufficient dimension reduction methods are developed for data with a continuous response and may have an unsatisfactory performance for the categorical response, especially for the binary-response. To address this issue, we propose a novel estimation method of sufficient dimension reduction subspace (SDR subspace) using optimal transport. The proposed method, named principal optimal transport direction (POTD), estimates the basis of the SDR subspace using the principal directions of the optimal transport coupling between the data respecting different response categories. The proposed method also reveals the relationship among three seemingly irrelevant topics, i.e., sufficient dimension reduction, support vector machine, and optimal transport. We study the asymptotic properties of POTD and show that in the cases when the class labels contain no error, POTD estimates the SDR subspace exclusively. Empirical studies show POTD outperforms most of the state-of-the-art linear dimension reduction methods.

preprint2021arXiv

The Story in Your Eyes: An Individual-difference-aware Model for Cross-person Gaze Estimation

We propose a novel method on refining cross-person gaze prediction task with eye/face images only by explicitly modelling the person-specific differences. Specifically, we first assume that we can obtain some initial gaze prediction results with existing method, which we refer to as InitNet, and then introduce three modules, the Validity Module (VM), Self-Calibration (SC) and Person-specific Transform (PT)) Module. By predicting the reliability of current eye/face images, our VM is able to identify invalid samples, e.g. eye blinking images, and reduce their effects in our modelling process. Our SC and PT module then learn to compensate for the differences on valid samples only. The former models the translation offsets by bridging the gap between initial predictions and dataset-wise distribution. And the later learns more general person-specific transformation by incorporating the information from existing initial predictions of the same person. We validate our ideas on three publicly available datasets, EVE, XGaze and MPIIGaze and demonstrate that our proposed method outperforms the SOTA methods significantly on all of them, e.g. respectively 21.7%, 36.0% and 32.9% relative performance improvements. We won the GAZE 2021 Competition on the EVE dataset. Our code can be found here https://github.com/bjj9/EVE_SCPT.

preprint2020arXiv

Cross-modal Subspace Learning via Kernel Correlation Maximization and Discriminative Structure Preserving

The measure between heterogeneous data is still an open problem. Many research works have been developed to learn a common subspace where the similarity between different modalities can be calculated directly. However, most of existing works focus on learning a latent subspace but the semantically structural information is not well preserved. Thus, these approaches cannot get desired results. In this paper, we propose a novel framework, termed Cross-modal subspace learning via Kernel correlation maximization and Discriminative structure-preserving (CKD), to solve this problem in two aspects. Firstly, we construct a shared semantic graph to make each modality data preserve the neighbor relationship semantically. Secondly, we introduce the Hilbert-Schmidt Independence Criteria (HSIC) to ensure the consistency between feature-similarity and semantic-similarity of samples. Our model not only considers the inter-modality correlation by maximizing the kernel correlation but also preserves the semantically structural information within each modality. The extensive experiments are performed to evaluate the proposed framework on the three public datasets. The experimental results demonstrated that the proposed CKD is competitive compared with the classic subspace learning methods.

preprint2020arXiv

Deep Fusion Siamese Network for Automatic Kinship Verification

Automatic kinship verification aims to determine whether some individuals belong to the same family. It is of great research significance to help missing persons reunite with their families. In this work, the challenging problem is progressively addressed in two respects. First, we propose a deep siamese network to quantify the relative similarity between two individuals. When given two input face images, the deep siamese network extracts the features from them and fuses these features by combining and concatenating. Then, the fused features are fed into a fully-connected network to obtain the similarity score between two faces, which is used to verify the kinship. To improve the performance, a jury system is also employed for multi-model fusion. Second, two deep siamese networks are integrated into a deep triplet network for tri-subject (i.e., father, mother and child) kinship verification, which is intended to decide whether a child is related to a pair of parents or not. Specifically, the obtained similarity scores of father-child and mother-child are weighted to generate the parent-child similarity score for kinship verification. Recognizing Families In the Wild (RFIW) is a challenging kinship recognition task with multiple tracks, which is based on Families in the Wild (FIW), a large-scale and comprehensive image database for automatic kinship recognition. The Kinship Verification (track I) and Tri-Subject Verification (track II) are supported during the ongoing RFIW2020 Challenge. Our team (ustc-nelslip) ranked 1st in track II, and 3rd in track I. The code is available at https://github.com/gniknoil/FG2020-kinship.

preprint2020arXiv

Detecting Communities in Heterogeneous Multi-Relational Networks:A Message Passing based Approach

Community is a common characteristic of networks including social networks, biological networks, computer and information networks, to name a few. Community detection is a basic step for exploring and analysing these network data. Typically, homogenous network is a type of networks which consists of only one type of objects with one type of links connecting them. There has been a large body of developments in models and algorithms to detect communities over it. However, real-world networks naturally exhibit heterogeneous qualities appearing as multiple types of objects with multi-relational links connecting them. Those heterogeneous information could facilitate the community detection for its constituent homogeneous networks, but has not been fully explored. In this paper, we exploit heterogeneous multi-relational networks (HMRNet) and propose an efficient message passing based algorithm to simultaneously detect communities for all homogeneous networks. Specifically, an HMRNet is reorganized into a hierarchical structure with homogeneous networks as its layers and heterogeneous links connecting them. To detect communities in such an HMRNet, the problem is formulated as a maximum a posterior (MAP) over a factor graph. Finally a message passing based algorithm is derived to find a best solution of the MAP problem. Evaluation on both synthetic and real-world networks confirms the effectiveness of the proposed method.

preprint2020arXiv

Learning Discriminative Hashing Codes for Cross-Modal Retrieval based on Multi-view Features

Hashing techniques have been applied broadly in retrieval tasks due to their low storage requirements and high speed of processing. Many hashing methods based on a single view have been extensively studied for information retrieval. However, the representation capacity of a single view is insufficient and some discriminative information is not captured, which results in limited improvement. In this paper, we employ multiple views to represent images and texts for enriching the feature information. Our framework exploits the complementary information among multiple views to better learn the discriminative compact hash codes. A discrete hashing learning framework that jointly performs classifier learning and subspace learning is proposed to complete multiple search tasks simultaneously. Our framework includes two stages, namely a kernelization process and a quantization process. Kernelization aims to find a common subspace where multi-view features can be fused. The quantization stage is designed to learn discriminative unified hashing codes. Extensive experiments are performed on single-label datasets (WiKi and MMED) and multi-label datasets (MIRFlickr and NUS-WIDE) and the experimental results indicate the superiority of our method compared with the state-of-the-art methods.

preprint2020arXiv

Learning Domain-invariant Graph for Adaptive Semi-supervised Domain Adaptation with Few Labeled Source Samples

Domain adaptation aims to generalize a model from a source domain to tackle tasks in a related but different target domain. Traditional domain adaptation algorithms assume that enough labeled data, which are treated as the prior knowledge are available in the source domain. However, these algorithms will be infeasible when only a few labeled data exist in the source domain, and thus the performance decreases significantly. To address this challenge, we propose a Domain-invariant Graph Learning (DGL) approach for domain adaptation with only a few labeled source samples. Firstly, DGL introduces the Nystrom method to construct a plastic graph that shares similar geometric property as the target domain. And then, DGL flexibly employs the Nystrom approximation error to measure the divergence between plastic graph and source graph to formalize the distribution mismatch from the geometric perspective. Through minimizing the approximation error, DGL learns a domain-invariant geometric graph to bridge source and target domains. Finally, we integrate the learned domain-invariant graph with the semi-supervised learning and further propose an adaptive semi-supervised model to handle the cross-domain problems. The results of extensive experiments on popular datasets verify the superiority of DGL, especially when only a few labeled source samples are available.

preprint2020arXiv

Repulsive Mixture Models of Exponential Family PCA for Clustering

The mixture extension of exponential family principal component analysis (EPCA) was designed to encode much more structural information about data distribution than the traditional EPCA does. For example, due to the linearity of EPCA's essential form, nonlinear cluster structures cannot be easily handled, but they are explicitly modeled by the mixing extensions. However, the traditional mixture of local EPCAs has the problem of model redundancy, i.e., overlaps among mixing components, which may cause ambiguity for data clustering. To alleviate this problem, in this paper, a repulsiveness-encouraging prior is introduced among mixing components and a diversified EPCA mixture (DEPCAM) model is developed in the Bayesian framework. Specifically, a determinantal point process (DPP) is exploited as a diversity-encouraging prior distribution over the joint local EPCAs. As required, a matrix-valued measure for L-ensemble kernel is designed, within which, $\ell_1$ constraints are imposed to facilitate selecting effective PCs of local EPCAs, and angular based similarity measure are proposed. An efficient variational EM algorithm is derived to perform parameter learning and hidden variable inference. Experimental results on both synthetic and real-world datasets confirm the effectiveness of the proposed method in terms of model parsimony and generalization ability on unseen test data.

preprint2020arXiv

Retrieval of Family Members Using Siamese Neural Network

Retrieval of family members in the wild aims at finding family members of the given subject in the dataset, which is useful in finding the lost children and analyzing the kinship. However, due to the diversity in age, gender, pose and illumination of the collected data, this task is always challenging. To solve this problem, we propose our solution with deep Siamese neural network. Our solution can be divided into two parts: similarity computation and ranking. In training procedure, the Siamese network firstly takes two candidate images as input and produces two feature vectors. And then, the similarity between the two vectors is computed with several fully connected layers. While in inference procedure, we try another similarity computing method by dropping the followed several fully connected layers and directly computing the cosine similarity of the two feature vectors. After similarity computation, we use the ranking algorithm to merge the similarity scores with the same identity and output the ordered list according to their similarities. To gain further improvement, we try different combinations of backbones, training methods and similarity computing methods. Finally, we submit the best combination as our solution and our team(ustc-nelslip) obtains favorable result in the track3 of the RFIW2020 challenge with the first runner-up, which verifies the effectiveness of our method. Our code is available at: https://github.com/gniknoil/FG2020-kinship

preprint2020arXiv

Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs

Face photo-sketch synthesis aims at generating a facial sketch/photo conditioned on a given photo/sketch. It is of wide applications including digital entertainment and law enforcement. Precisely depicting face photos/sketches remains challenging due to the restrictions on structural realism and textural consistency. While existing methods achieve compelling results, they mostly yield blurred effects and great deformation over various facial components, leading to the unrealistic feeling of synthesized images. To tackle this challenge, in this work, we propose to use the facial composition information to help the synthesis of face sketch/photo. Specially, we propose a novel composition-aided generative adversarial network (CA-GAN) for face photo-sketch synthesis. In CA-GAN, we utilize paired inputs including a face photo/sketch and the corresponding pixel-wise face labels for generating a sketch/photo. In addition, to focus training on hard-generated components and delicate facial structures, we propose a compositional reconstruction loss. Finally, we use stacked CA-GANs (SCA-GAN) to further rectify defects and add compelling details. Experimental results show that our method is capable of generating both visually comfortable and identity-preserving face sketches/photos over a wide range of challenging data. Our method achieves the state-of-the-art quality, reducing best previous Frechet Inception distance (FID) by a large margin. Besides, we demonstrate that the proposed method is of considerable generalization ability. We have made our code and results publicly available: https://fei-hdu.github.io/ca-gan/.

preprint2020arXiv

Weakly-Supervised Multi-Level Attentional Reconstruction Network for Grounding Textual Queries in Videos

The task of temporally grounding textual queries in videos is to localize one video segment that semantically corresponds to the given query. Most of the existing approaches rely on segment-sentence pairs (temporal annotations) for training, which are usually unavailable in real-world scenarios. In this work we present an effective weakly-supervised model, named as Multi-Level Attentional Reconstruction Network (MARN), which only relies on video-sentence pairs during the training stage. The proposed method leverages the idea of attentional reconstruction and directly scores the candidate segments with the learnt proposal-level attentions. Moreover, another branch learning clip-level attention is exploited to refine the proposals at both the training and testing stage. We develop a novel proposal sampling mechanism to leverage intra-proposal information for learning better proposal representation and adopt 2D convolution to exploit inter-proposal clues for learning reliable attention map. Experiments on Charades-STA and ActivityNet-Captions datasets demonstrate the superiority of our MARN over the existing weakly-supervised methods.

preprint2019arXiv

Enhanced block sparse signal recovery based on $q$-ratio block constrained minimal singular values

In this paper we introduce the $q$-ratio block constrained minimal singular values (BCMSV) as a new measure of measurement matrix in compressive sensing of block sparse/compressive signals and present an algorithm for computing this new measure. Both the mixed $\ell_2/\ell_q$ and the mixed $\ell_2/\ell_1$ norms of the reconstruction errors for stable and robust recovery using block Basis Pursuit (BBP), the block Dantzig selector (BDS) and the group lasso in terms of the $q$-ratio BCMSV are investigated. We establish a sufficient condition based on the $q$-ratio block sparsity for the exact recovery from the noise free BBP and developed a convex-concave procedure to solve the corresponding non-convex problem in the condition. Furthermore, we prove that for sub-Gaussian random matrices, the $q$-ratio BCMSV is bounded away from zero with high probability when the number of measurements is reasonably large. Numerical experiments are implemented to illustrate the theoretical results. In addition, we demonstrate that the $q$-ratio BCMSV based error bounds are tighter than the block restricted isotropic constant based bounds.