Researcher profile

Xi Chen

Xi Chen contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate

Machine Learning quant-ph Computer Vision Artificial Intelligence Methodology Computation and Language Data Structures and Algorithms cond-mat.mtrl-sci physics.comp-ph cond-mat.quant-gas Information Retrieval physics.optics astro-ph.GA math.ST Statistics Theory Computer Science and Game Theory cond-mat.mes-hall cond-mat.str-el eess.AS eess.IV eess.SP physics.app-ph physics.atom-ph Sound astro-ph.IM astro-ph.SR Computation Computational Complexity cond-mat.dis-nn cond-mat.supr-con Cryptography and Security Databases econ.EM eess.SY Emerging Technologies Information Theory math.IT math.NA math.OC math.PR nlin.PS Numerical Analysis Robotics Systems and Control astro-ph.CO Biological Physics cond-mat.soft cond-mat.stat-mech cs.CY Discrete Mathematics Distributed, Parallel, and Cluster Computing Graphics hep-ph Human-Computer Interaction math.AG math.CO Neural and Evolutionary Computing nlin.SI physics.chem-ph q-fin.GN Quantitative Methods Social and Information Networks

Trust snapshot

Quick read

Trust 21 - EmergingVerification L1Unclaimed author

132works

0followers

62topics

4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2026arXiv

PanoWorld: Towards Spatial Supersensing in 360$^\circ$ Panorama World

Multimodal large laboratory models (MLLMs) still struggle with spatial understanding under the dominant perspective-image paradigm, which inherits the narrow field of view of human-like perception. For navigation, robotic search, and 3D scene understanding, 360-degree panoramic sensing offers a form of supersensing by capturing the entire surrounding environment at once. However, existing MLLM pipelines typically decompose panoramas into multiple perspective views, leaving the spherical structure of equirectangular projection (ERP) largely implicit. In this paper, we study pano-native understanding, which requires an MLLM to reason over an ERP panorama as a continuous, observer-centered space. To this end, we first define the key abilities for pano-native understanding, including semantic anchoring, spherical localization, reference-frame transformation, and depth-aware 3D spatial reasoning. We then build a large-scale metadata construction pipeline that converts mixed-source ERP panoramas into geometry-aware, language-grounded, and depth-aware supervision, and instantiate these signals as capability-aligned instruction tuning data. On the model side, we introduce PanoWorld with Spherical Spatial Cross-Attention, which injects spherical geometry into the visual stream. We further construct PanoSpace-Bench, a diagnostic benchmark for evaluating ERP-native spatial reasoning. Experiments show that PanoWorld substantially outperforms both proprietary and open-source baselines on PanoSpace-Bench, H* Bench, and R2R-CE Val-Unseen benchmarks. These results demonstrate that robust panoramic reasoning requires dedicated pano-native supervision and geometry-aware model adaptation. All source code and proposed data will be publicly released.

preprint2025arXiv

Demonstration of Hardware Efficient Photonic Variational Quantum Algorithm

Quantum computing has brought a paradigm change in computer science, where non-classical technologies have promised to outperform their classical counterpart. Such an advantage was only demonstrated for tasks without practical applications, still out of reach for the state-of-art quantum technologies. In this context, a promising strategy to find practical use of quantum computers is to exploit hybrid quantum-classical models, where a quantum device estimates a hard-to-compute quantity, while a classical optimizer trains the parameters of the model. In this work, we demonstrate that single photons and linear optical networks are sufficient for implementing Variational Quantum Algorithms, when the problem specification, or ansatz, is tailored to this specific platform. We show this by a proof-of-principle demonstration of a variational approach to tackle an instance of a factorization task, whose solution is encoded in the ground state of a suitable Hamiltonian. This work which combines Variational Quantum Algorithms with hardware efficient ansatzes for linear-optics networks showcases a promising pathway towards practical applications for photonic quantum platforms.

preprint2025arXiv

OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditions

Existing feedforward subject-driven video customization methods mainly study single-subject scenarios due to the difficulty of constructing multi-subject training data pairs. Another challenging problem that how to use the signals such as depth, mask, camera, and text prompts to control and edit the subject in the customized video is still less explored. In this paper, we first propose a data construction pipeline, VideoCus-Factory, to produce training data pairs for multi-subject customization from raw videos without labels and control signals such as depth-to-video and mask-to-video pairs. Based on our constructed data, we develop an Image-Video Transfer Mixed (IVTM) training with image editing data to enable instructive editing for the subject in the customized video. Then we propose a diffusion Transformer framework, OmniVCus, with two embedding mechanisms, Lottery Embedding (LE) and Temporally Aligned Embedding (TAE). LE enables inference with more subjects by using the training subjects to activate more frame embeddings. TAE encourages the generation process to extract guidance from temporally aligned control signals by assigning the same frame embeddings to the control and noise tokens. Experiments demonstrate that our method significantly surpasses state-of-the-art methods in both quantitative and qualitative evaluations. Video demos are at our project page: https://caiyuanhao1998.github.io/project/OmniVCus/. Our code, models, data are released at https://github.com/caiyuanhao1998/Open-OmniVCus

preprint2024arXiv

Depth Map Denoising Network and Lightweight Fusion Network for Enhanced 3D Face Recognition

With the increasing availability of consumer depth sensors, 3D face recognition (FR) has attracted more and more attention. However, the data acquired by these sensors are often coarse and noisy, making them impractical to use directly. In this paper, we introduce an innovative Depth map denoising network (DMDNet) based on the Denoising Implicit Image Function (DIIF) to reduce noise and enhance the quality of facial depth images for low-quality 3D FR. After generating clean depth faces using DMDNet, we further design a powerful recognition network called Lightweight Depth and Normal Fusion network (LDNFNet), which incorporates a multi-branch fusion block to learn unique and complementary features between different modalities such as depth and normal images. Comprehensive experiments conducted on four distinct low-quality databases demonstrate the effectiveness and robustness of our proposed methods. Furthermore, when combining DMDNet and LDNFNet, we achieve state-of-the-art results on the Lock3DFace database.

preprint2024arXiv

Transfer the linguistic representations from TTS to accent conversion with non-parallel data

Accent conversion aims to convert the accent of a source speech to a target accent, meanwhile preserving the speaker's identity. This paper introduces a novel non-autoregressive framework for accent conversion that learns accent-agnostic linguistic representations and employs them to convert the accent in the source speech. Specifically, the proposed system aligns speech representations with linguistic representations obtained from Text-to-Speech (TTS) systems, enabling training of the accent voice conversion model on non-parallel data. Furthermore, we investigate the effectiveness of a pretraining strategy on native data and different acoustic features within our proposed framework. We conduct a comprehensive evaluation using both subjective and objective metrics to assess the performance of our approach. The evaluation results highlight the benefits of the pretraining strategy and the incorporation of richer semantic features, resulting in significantly enhanced audio quality and intelligibility.

preprint2023arXiv

A Lightweight Clustering Framework for Unsupervised Semantic Segmentation

Unsupervised semantic segmentation aims to categorize each pixel in an image into a corresponding class without the use of annotated data. It is a widely researched area as obtaining labeled datasets is expensive. While previous works in the field have demonstrated a gradual improvement in model accuracy, most required neural network training. This made segmentation equally expensive, especially when dealing with large-scale datasets. We thus propose a lightweight clustering framework for unsupervised semantic segmentation. We discovered that attention features of the self-supervised Vision Transformer exhibit strong foreground-background differentiability. Therefore, clustering can be employed to effectively separate foreground and background image patches. In our framework, we first perform multilevel clustering across the Dataset-level, Category-level, and Image-level, and maintain consistency throughout. Then, the binary patch-level pseudo-masks extracted are upsampled, refined and finally labeled. Furthermore, we provide a comprehensive analysis of the self-supervised Vision Transformer features and a detailed comparison between DINO and DINOv2 to justify our claims. Our framework demonstrates great promise in unsupervised semantic segmentation and achieves state-of-the-art results on PASCAL VOC and MS COCO datasets.

preprint2023arXiv

High precision atom interferometer-based dynamic gravimeter measurement by eliminating the cross-coupling effect

A dynamic gravimeter with an atomic interferometer (AI) can perform absolute gravity measurements with high precision. AI-based dynamic gravity measurement is a type of joint measurement that uses AI sensors and a classical accelerometer. The coupling of the two sensors may degrade the measurement precision. In this study, we analyzed the cross-coupling effect and introduced a recovery vector to suppress this effect. We improved the phase noise of the interference fringe by a factor of 1.9 by performing marine gravity measurements using an AI-based gravimeter and optimizing the recovery vector. Marine gravity measurements were performed, and high gravity measurement precision was achieved. The external and inner coincidence accuracies of the gravity measurement are 0.42 mGal and 0.46 mGal, which were improved by factors of 4.18 and 4.21 by optimizing the cross-coupling effect.

preprint2023arXiv

LaFFi: Leveraging Hybrid Natural Language Feedback for Fine-tuning Language Models

Fine-tuning Large Language Models (LLMs) adapts a trained model to specific downstream tasks, significantly improving task-specific performance. Supervised Fine-Tuning (SFT) is a common approach, where an LLM is trained to produce desired answers. However, LLMs trained with SFT sometimes make simple mistakes and result in hallucinations on reasoning tasks such as question-answering. Without external feedback, it is difficult for SFT to learn a good mapping between the question and the desired answer, especially with a small dataset. This paper introduces an alternative to SFT called Natural Language Feedback for Finetuning LLMs (LaFFi). LaFFi has LLMs directly predict the feedback they will receive from an annotator. We find that requiring such reflection can significantly improve the accuracy in in-domain question-answering tasks, providing a promising direction for the application of natural language feedback in the realm of SFT LLMs. Additional ablation studies show that the portion of human-annotated data in the annotated datasets affects the fine-tuning performance.

preprint2023arXiv

Shortcuts to Adiabatic Soliton Compression in Active Nonlinear Kerr Media

We implement variational shortcuts to adiabaticity for optical pulse compression in an active nonlinear Kerr medium with distributed amplification and spatially varying dispersion and nonlinearity. Starting with the hyperbolic secant ansatz, we employ a variational approximation to systematically derive dynamical equations, establishing analytical relationships linking the amplitude, width, and chirp of the pulse. Through the inverse engineering approach, we manipulate the distributed gain/loss, nonlinearity and dispersion profiles to efficiently compress the optical pulse over a reduced distance with high fidelity. In addition, we explore the dynamical stability of the system to illustrate the advantage of our protocol over conventional adiabatic approaches. Finally, we analyze the impact of tailored higher-order dispersion on soliton self-compression and derive physical constraints on the final soliton width for the complementary case of soliton expansion. The broader implications of our findings extend beyond optical systems, encompassing areas such as cold-atom and magnetic systems highlighting the versatility and relevance of our approach in various physical contexts.

preprint2022arXiv

Accelerating Adaptive Cubic Regularization of Newton's Method via Random Sampling

In this paper, we consider an unconstrained optimization model where the objective is a sum of a large number of possibly nonconvex functions, though overall the objective is assumed to be smooth and convex. Our bid to solving such model uses the framework of cubic regularization of Newton's method. As well known, the crux in cubic regularization is its utilization of the Hessian information, which may be computationally expensive for large-scale problems. To tackle this, we resort to approximating the Hessian matrix via sub-sampling. In particular, we propose to compute an approximated Hessian matrix by either \textit{uniformly}\/ or \textit{non-uniformly}\/ sub-sampling the components of the objective. Based upon such sampling strategy, we develop accelerated adaptive cubic regularization approaches and provide theoretical guarantees on global iteration complexity of $O(ε^{-1/3})$ with high probability, which matches that of the original accelerated cubic regularization methods \cite{Jiang-2017-Unified} using the \textit{full}\/ Hessian information. Interestingly, we show that in the worst case scenario our algorithm still achieves an $O\left(\log(ε^{-1})ε^{-5/6}\right)$ iteration complexity bound. The performances of the proposed methods on the regularized logistic regression problems show a clear effect of acceleration in terms of the epoch counts on several real data sets.

preprint2022arXiv

Active Learning for Contextual Search with Binary Feedbacks

In this paper, we study the learning problem in contextual search, which is motivated by applications such as first-price auction, personalized medicine experiments, and feature-based pricing experiments. In particular, for a sequence of arriving context vectors, with each context associated with an underlying value, the decision-maker either makes a query at a certain point or skips the context. The decision-maker will only observe the binary feedback on the relationship between the query point and the value associated with the context. We study a PAC learning setting, where the goal is to learn the underlying mean value function in context with a minimum number of queries. To address this challenge, we propose a tri-section search approach combined with a margin-based active learning method. We show that the algorithm only needs to make $O(1/\varepsilon^2)$ queries to achieve an $ε$-estimation accuracy. This sample complexity significantly reduces the required sample complexity in the passive setting, at least $Ω(1/\varepsilon^4)$.

preprint2022arXiv

All You May Need for VQA are Image Captions

Visual Question Answering (VQA) has benefited from increasingly sophisticated models, but has not enjoyed the same level of engagement in terms of data creation. In this paper, we propose a method that automatically derives VQA examples at volume, by leveraging the abundance of existing image-caption annotations combined with neural models for textual question generation. We show that the resulting data is of high-quality. VQA models trained on our data improve state-of-the-art zero-shot accuracy by double digits and achieve a level of robustness that lacks in the same model trained on human-annotated VQA data.

preprint2022arXiv

AutoMO-Mixer: An automated multi-objective Mixer model for balanced, safe and robust prediction in medicine

Accurately identifying patient's status through medical images plays an important role in diagnosis and treatment. Artificial intelligence (AI), especially the deep learning, has achieved great success in many fields. However, more reliable AI model is needed in image guided diagnosis and therapy. To achieve this goal, developing a balanced, safe and robust model with a unified framework is desirable. In this study, a new unified model termed as automated multi-objective Mixer (AutoMO-Mixer) model was developed, which utilized a recent developed multiple layer perceptron Mixer (MLP-Mixer) as base. To build a balanced model, sensitivity and specificity were considered as the objective functions simultaneously in training stage. Meanwhile, a new evidential reasoning based on entropy was developed to achieve a safe and robust model in testing stage. The experiment on an optical coherence tomography dataset demonstrated that AutoMO-Mixer can obtain safer, more balanced, and robust results compared with MLP-Mixer and other available models.

preprint2022arXiv

Bayesian posterior repartitioning for nested sampling

Priors in Bayesian analyses often encode informative domain knowledge that can be useful in making the inference process more efficient. Occasionally, however, priors may be unrepresentative of the parameter values for a given dataset, which can result in inefficient parameter space exploration, or even incorrect inferences, particularly for nested sampling (NS) algorithms. Simply broadening the prior in such cases may be inappropriate or impossible in some applications. Hence our previous solution to this problem, known as posterior repartitioning (PR), redefines the prior and likelihood while keeping their product fixed, so that the posterior inferences and evidence estimates remain unchanged, but the efficiency of the NS process is significantly increased. In its most practical form, PR raises the prior to some power beta, which is introduced as an auxiliary variable that must be determined on a case-by-case basis, usually by lowering beta from unity according to some pre-defined `annealing schedule' until the resulting inferences converge to a consistent solution. Here we present a very simple yet powerful alternative Bayesian approach, in which beta is instead treated as a hyperparameter that is inferred from the data alongside the original parameters of the problem, and then marginalised over to obtain the final inference. We show through numerical examples that this Bayesian PR (BPR) method provides a very robust, self-adapting and computationally efficient `hands-off' solution to the problem of unrepresentative priors in Bayesian inference using NS. Moreover, unlike the original PR method, we show that even for representative priors BPR has a negligible computational overhead relative to standard nesting sampling, which suggests that it should be used as the default in all NS analyses.

preprint2022arXiv

Collaborative learning of images and geometrics for predicting isocitrate dehydrogenase status of glioma

The isocitrate dehydrogenase (IDH) gene mutation status is an important biomarker for glioma patients. The gold standard of IDH mutation detection requires tumour tissue obtained via invasive approaches and is usually expensive. Recent advancement in radiogenomics provides a non-invasive approach for predicting IDH mutation based on MRI. Meanwhile, tumor geometrics encompass crucial information for tumour phenotyping. Here we propose a collaborative learning framework that learns both tumor images and tumor geometrics using convolutional neural networks (CNN) and graph neural networks (GNN), respectively. Our results show that the proposed model outperforms the baseline model of 3D-DenseNet121. Further, the collaborative learning model achieves better performance than either the CNN or the GNN alone. The model interpretation shows that the CNN and GNN could identify common and unique regions of interest for IDH mutation prediction. In conclusion, collaborating image and geometric learners provides a novel approach for predicting genotype and characterising glioma.

preprint2022arXiv

Context-Based Dynamic Pricing with Online Clustering

We consider a context-based dynamic pricing problem of online products, which have low sales. Sales data from Alibaba, a major global online retailer, illustrate the prevalence of low-sale products. For these products, existing single-product dynamic pricing algorithms do not work well due to insufficient data samples. To address this challenge, we propose pricing policies that concurrently perform clustering over product demand and set individual pricing decisions on the fly. By clustering data and identifying products that have similar demand patterns, we utilize sales data from products within the same cluster to improve demand estimation for better pricing decisions. We evaluate the algorithms using regret, and the result shows that when product demand functions come from multiple clusters, our algorithms significantly outperform traditional single-product pricing policies. Numerical experiments using a real dataset from Alibaba demonstrate that the proposed policies, compared with several benchmark policies, increase the revenue. The results show that online clustering is an effective approach to tackling dynamic pricing problems associated with low-sale products.

preprint2022arXiv

Coverage Optimization of Camera Network for Continuous Deformable Object

In this paper, a deformable object is considered for cameras deployment with the aim of visual coverage. The object contour is discretized into sampled points as meshes, and the deformation is represented as continuous trajectories for the sampled points. To reduce the computational complexity, some feature points are carefully selected representing the continuous deformation process, and the visual coverage for the deformable object is transferred to cover the specific feature points. In particular, the vertexes of a rectangle that can contain the entire deformation trajectory of every sampled point on the object contour are chosen as the feature points. An improved wolf pack algorithm is then proposed to solve the optimization problem. Finally, simulation results are given to demonstrate the effectiveness of the proposed deployment method of camera network.

preprint2022arXiv

Development of a compact high-resolution absolute gravity gradiometer based on atom interferometers

We present a compact high-resolution gravity gradiometer based on dual Rb-85 atom interferometers using stimulated Raman transitions. A baseline L=44.5 cm and an interrogation time T=130 ms are realized in a sensor head with volume of less than 100 liters. Experimental parameters are optimized to improve the short-term sensitivity while a rejection algorithm relying on inversion of the Raman wave vector is implemented to improve the long-term stability. After an averaging time of 17000 s, a phase resolution of 104 μrad is achieved, which corresponds to a gravity gradient resolution of 0.86 E. As far as we know, this is the sub-E atom gravity gradiometer with the highest level of compactness to date. After the evaluation and correction of system errors induced by light shift, residual Zeeman shift, Coriolis effect and self-attraction effect, the instrument serves as an absolute gravity gradiometer and with it the local gravity gradient is measured to be 3114 (53) E.

preprint2022arXiv

Digitized-counterdiabatic quantum approximate optimization algorithm

The quantum approximate optimization algorithm (QAOA) has proved to be an effective classical-quantum algorithm serving multiple purposes, from solving combinatorial optimization problems to finding the ground state of many-body quantum systems. Since QAOA is an ansatz-dependent algorithm, there is always a need to design ansatz for better optimization. To this end, we propose a digitized version of QAOA enhanced via the use of shortcuts to adiabaticity. Specifically, we use a counterdiabatic (CD) driving term to design a better ansatz, along with the Hamiltonian and mixing terms, enhancing the global performance. We apply our digitized-counterdiabatic QAOA to Ising models, classical optimization problems, and the P-spin model, demonstrating that it outperforms standard QAOA in all cases we study.

preprint2022arXiv

Digitized-Counterdiabatic Quantum Optimization

We propose digitized-counterdiabatic quantum optimization (DCQO) to achieve polynomial enhancement over adiabatic quantum optimization for the general Ising spin-glass model, which includes the whole class of combinatorial optimization problems. This is accomplished via the digitization of adiabatic quantum algorithms that are catalysed by the addition of non-stoquastic counterdiabatic terms. The latter are suitably chosen, not only for escaping classical simulability, but also for speeding up the performance. Finding the ground state of a general Ising spin-glass Hamiltonian is used to illustrate that the inclusion of k-local non-stoquastic counterdiabatic terms can always outperform the traditional adiabatic quantum optimization with stoquastic Hamiltonians. In particular, we show that a polynomial enhancement in the ground-state success probability can be achieved for a finite-time evolution, even with the simplest 2-local counterdiabatic terms. Furthermore, the considered digitization process, within the gate-based quantum computing paradigm, provides the flexibility to introduce arbitrary non-stoquastic interactions. Along these lines, using our proposed paradigm on current NISQ computers, quantum speed-up may be reached to find approximate solutions for NP-complete and NP-hard optimization problems. We expect DCQO to become a fast-lane paradigm towards quantum advantage in the NISQ era.

preprint2022arXiv

Dynamic Car Dispatching and Pricing: Revenue and Fairness for Ridesharing Platforms

A major challenge for ridesharing platforms is to guarantee profit and fairness simultaneously, especially in the presence of misaligned incentives of drivers and riders. We focus on the dispatching-pricing problem to maximize the total revenue while keeping both drivers and riders satisfied. We study the computational complexity of the problem, provide a novel two-phased pricing solution with revenue and fairness guarantees, extend it to stochastic settings and develop a dynamic (a.k.a., learning-while-doing) algorithm that actively collects data to learn the demand distribution during the scheduling process. We also conduct extensive experiments to demonstrate the effectiveness of our algorithms.

preprint2022arXiv

Efficient broadband frequency conversion via shortcut to adiabaticity

The method of adiabatic frequency conversion, in analogy with the two level atomic system, has been put forward recently and verified experimentally to achieve robust frequency mixing processes such as sum and difference frequency generation. Here we present a comparative study of efficient frequency mixing using various techniques of shortcuts to adiabaticity (STA) such as counter-diabatic driving and invariant-based inverse engineering. We show that, it is possible to perform sum frequency generation by properly designing the poling structure of a periodically poled crystal and the coupling between the input lights and the crystal. The required crystal length for frequency conversion is significantly decreases beyond the adiabatic limit. Our approach significantly improves the robustness of the process against the variation in temperature as well as the signal frequency. By introducing a single parameter control technique with constant coupling and combining with the inverse engineering, perturbation theory and optimal control, we show that the phase mismatch can be further optimized with respect to the fluctuations of input wavelength and crystal temperature that results into a novel experimentally realizable mixing scheme.

preprint2022arXiv

Encoder-Decoder Architecture for Supervised Dynamic Graph Learning: A Survey

In recent years, the prevalent online services generate a sheer volume of user activity data. Service providers collect these data in order to perform client behavior analysis, and offer better and more customized services. Majority of these data can be modeled and stored as graph, such as the social graph in Facebook, user-video interaction graph in Youtube. These graphs need to evolve over time to capture the dynamics in the real world, leading to the invention of dynamic graphs. However, the temporal information embedded in the dynamic graphs brings new challenges in analyzing and deploying them. Events staleness, temporal information learning and explicit time dimension usage are some example challenges in dynamic graph learning. In order to offer a convenient reference to both the industry and academia, this survey presents the Three Stages Recurrent Temporal Learning Framework based on dynamic graph evolution theories, so as to interpret the learning of temporal information with a generalized framework. Under this framework, this survey categories and reviews different learnable encoder-decoder architectures for supervised dynamic graph learning. We believe that this survey could supply useful guidelines to researchers and engineers in finding suitable graph structures for their dynamic learning tasks.

preprint2022arXiv

Finding Influential Instances for Distantly Supervised Relation Extraction

Distant supervision (DS) is a strong way to expand the datasets for enhancing relation extraction (RE) models but often suffers from high label noise. Current works based on attention, reinforcement learning, or GAN are black-box models so they neither provide meaningful interpretation of sample selection in DS nor stability on different domains. On the contrary, this work proposes a novel model-agnostic instance sampling method for DS by influence function (IF), namely REIF. Our method identifies favorable/unfavorable instances in the bag based on IF, then does dynamic instance sampling. We design a fast influence sampling algorithm that reduces the computational complexity from $\mathcal{O}(mn)$ to $\mathcal{O}(1)$, with analyzing its robustness on the selected sampling function. Experiments show that by simply sampling the favorable instances during training, REIF is able to win over a series of baselines that have complicated architectures. We also demonstrate that REIF can support interpretable instance selection.

preprint2022arXiv

Fixed-Support Wasserstein Barycenters: Computational Hardness and Fast Algorithm

We study the fixed-support Wasserstein barycenter problem (FS-WBP), which consists in computing the Wasserstein barycenter of $m$ discrete probability measures supported on a finite metric space of size $n$. We show first that the constraint matrix arising from the standard linear programming (LP) representation of the FS-WBP is \textit{not totally unimodular} when $m \geq 3$ and $n \geq 3$. This result resolves an open question pertaining to the relationship between the FS-WBP and the minimum-cost flow (MCF) problem since it proves that the FS-WBP in the standard LP form is not an MCF problem when $m \geq 3$ and $n \geq 3$. We also develop a provably fast \textit{deterministic} variant of the celebrated iterative Bregman projection (IBP) algorithm, named \textsc{FastIBP}, with a complexity bound of $\tilde{O}(mn^{7/3}\varepsilon^{-4/3})$, where $\varepsilon \in (0, 1)$ is the desired tolerance. This complexity bound is better than the best known complexity bound of $\tilde{O}(mn^2\varepsilon^{-2})$ for the IBP algorithm in terms of $\varepsilon$, and that of $\tilde{O}(mn^{5/2}\varepsilon^{-1})$ from accelerated alternating minimization algorithm or accelerated primal-dual adaptive gradient algorithm in terms of $n$. Finally, we conduct extensive experiments with both synthetic data and real images and demonstrate the favorable performance of the \textsc{FastIBP} algorithm in practice.

preprint2022arXiv

FocalClick: Towards Practical Interactive Image Segmentation

Interactive segmentation allows users to extract target masks by making positive/negative clicks. Although explored by many previous works, there is still a gap between academic approaches and industrial needs: first, existing models are not efficient enough to work on low power devices; second, they perform poorly when used to refine preexisting masks as they could not avoid destroying the correct part. FocalClick solves both issues at once by predicting and updating the mask in localized areas. For higher efficiency, we decompose the slow prediction on the entire image into two fast inferences on small crops: a coarse segmentation on the Target Crop, and a local refinement on the Focus Crop. To make the model work with preexisting masks, we formulate a sub-task termed Interactive Mask Correction, and propose Progressive Merge as the solution. Progressive Merge exploits morphological information to decide where to preserve and where to update, enabling users to refine any preexisting mask effectively. FocalClick achieves competitive results against SOTA methods with significantly smaller FLOPs. It also shows significant superiority when making corrections on preexisting masks. Code and data will be released at github.com/XavierCHEN34/ClickSEG

preprint2022arXiv

Improved Upper Bounds for Finding Tarski Fixed Points

We study the query complexity of finding a Tarski fixed point over the $k$-dimensional grid $\{1,\ldots,n\}^k$. Improving on the previous best upper bound of $\smash{O(\log^{\lceil 2k/3\rceil} n)}$ [FPS20], we give a new algorithm with query complexity $\smash{O(\log^{\lceil (k+1)/2\rceil} n)}$. This is based on a novel decomposition theorem about a weaker variant of the Tarski fixed point problem, where the input consists of a monotone function $f:[n]^k\rightarrow [n]^k$ and a monotone sign function $b:[n]^k\rightarrow \{-1,0,1\}$ and the goal is to find an $x\in [n]^k$ that satisfies $either$ $f(x)\preceq x$ and $b(x)\le 0$ $or$ $f(x)\succeq x$ and $b(x)\ge 0$.

preprint2022arXiv

Investigation of Singing Voice Separation for Singing Voice Detection in Polyphonic Music

Singing voice detection (SVD), to recognize vocal parts in the song, is an essential task in music information retrieval (MIR). The task remains challenging since singing voice varies and intertwines with the accompaniment music, especially for some complicated polyphonic music such as choral music recordings. To address this problem, we investigate singing voice detection while discarding the interference from the accompaniment. The proposed SVD has two steps: i. The singing voice separation (SVS) technique is first utilized to filter out the singing voice's potential part coarsely. ii. Upon the continuity of vocal in the time domain, Long-term Recurrent Convolutional Networks (LRCN) is used to learn compositional features. Moreover, to eliminate the outliers, we choose to use a median filter for time-domain smoothing. Experimental results show that the proposed method outperforms the existing state-of-the-art works on two public datasets, the Jamendo Corpus and the RWC pop dataset.

preprint2022arXiv

Latent-Variable Advantage-Weighted Policy Optimization for Offline RL

Offline reinforcement learning methods hold the promise of learning policies from pre-collected datasets without the need to query the environment for new transitions. This setting is particularly well-suited for continuous control robotic applications for which online data collection based on trial-and-error is costly and potentially unsafe. In practice, offline datasets are often heterogeneous, i.e., collected in a variety of scenarios, such as data from several human demonstrators or from policies that act with different purposes. Unfortunately, such datasets can exacerbate the distribution shift between the behavior policy underlying the data and the optimal policy to be learned, leading to poor performance. To address this challenge, we propose to leverage latent-variable policies that can represent a broader class of policy distributions, leading to better adherence to the training data distribution while maximizing reward via a policy over the latent variable. As we empirically show on a range of simulated locomotion, navigation, and manipulation tasks, our method referred to as latent-variable advantage-weighted policy optimization (LAPO), improves the average performance of the next best-performing offline reinforcement learning methods by 49% on heterogeneous datasets, and by 8% on datasets with narrow and biased distributions.

preprint2022arXiv

Leveraging Intrinsic Gradient Information for Further Training of Differentiable Machine Learning Models

Designing models that produce accurate predictions is the fundamental objective of machine learning (ML). This work presents methods demonstrating that when the derivatives of target variables (outputs) with respect to inputs can be extracted from processes of interest, e.g., neural networks (NN) based surrogate models, they can be leveraged to further improve the accuracy of differentiable ML models. This paper generalises the idea and provides practical methodologies that can be used to leverage gradient information (GI) across a variety of applications including: (1) Improving the performance of generative adversarial networks (GANs); (2) efficiently tuning NN model complexity; (3) regularising linear regressions. Numerical results show that GI can effective enhance ML models with existing datasets, demonstrating its value for a variety of applications.

preprint2022arXiv

Machine-learning assisted quantum control in random environment

Disorder in condensed matter and atomic physics is responsible for a great variety of fascinating quantum phenomena, which are still challenging for understanding, not to mention the relevant dynamical control. Here we introduce proof of the concept and analyze neural network-based machine learning algorithm for achieving feasible high-fidelity quantum control of a particle in random environment. To explicitly demonstrate its capabilities, we show that convolutional neural networks are able to solve this problem as they can recognize the disorder and, by supervised learning, further produce the policy for the efficient low-energy cost control of a quantum particle in a time-dependent random potential. We have shown that the accuracy of the proposed algorithm is enhanced by a higher-dimensional mapping of the disorder pattern and using two neural networks, each properly trained for the given task. The designed method, being computationally more efficient than the gradient-descent optimization, can be applicable to identify and control various noisy quantum systems on a heuristic basis.

preprint2022arXiv

Memory Bounds for Continual Learning

Continual learning, or lifelong learning, is a formidable current challenge to machine learning. It requires the learner to solve a sequence of $k$ different learning tasks, one after the other, while retaining its aptitude for earlier tasks; the continual learner should scale better than the obvious solution of developing and maintaining a separate learner for each of the $k$ tasks. We embark on a complexity-theoretic study of continual learning in the PAC framework. We make novel uses of communication complexity to establish that any continual learner, even an improper one, needs memory that grows linearly with $k$, strongly suggesting that the problem is intractable. When logarithmically many passes over the learning tasks are allowed, we provide an algorithm based on multiplicative weights update whose memory requirement scales well; we also establish that improper learning is necessary for such performance. We conjecture that these results may lead to new promising approaches to continual learning.

preprint2022arXiv

Meta-Learning Digitized-Counterdiabatic Quantum Optimization

Solving optimization tasks using variational quantum algorithms has emerged as a crucial application of the current noisy intermediate-scale quantum devices. However, these algorithms face several difficulties like finding suitable ansatz and appropriate initial parameters, among others. In this work, we tackle the problem of finding suitable initial parameters for variational optimization by employing a meta-learning technique using recurrent neural networks. We investigate this technique with the recently proposed digitized-counterdiabatic quantum approximate optimization algorithm (DC-QAOA) that utilizes counterdiabatic protocols to improve the state-of-the-art QAOA. The combination of meta learning and DC-QAOA enables us to find optimal initial parameters for different models, such as MaxCut problem and the Sherrington-Kirkpatrick model. Decreasing the number of iterations of optimization as well as enhancing the performance, our protocol designs short depth circuit ansatz with optimal initial parameters by incorporating shortcuts-to-adiabaticity principles into machine learning methods for the near-term devices.

preprint2022arXiv

Mitigating Data Heterogeneity in Federated Learning with Data Augmentation

Federated Learning (FL) is a prominent framework that enables training a centralized model while securing user privacy by fusing local, decentralized models. In this setting, one major obstacle is data heterogeneity, i.e., each client having non-identically and independently distributed (non-IID) data. This is analogous to the context of Domain Generalization (DG), where each client can be treated as a different domain. However, while many approaches in DG tackle data heterogeneity from the algorithmic perspective, recent evidence suggests that data augmentation can induce equal or greater performance. Motivated by this connection, we present federated versions of popular DG algorithms, and show that by applying appropriate data augmentation, we can mitigate data heterogeneity in the federated setting, and obtain higher accuracy on unseen clients. Equipped with data augmentation, we can achieve state-of-the-art performance using even the most basic Federated Averaging algorithm, with much sparser communication.

preprint2022arXiv

Molecular conformer search with low-energy latent space

Identifying low-energy conformers with quantum mechanical accuracy for molecules with many degrees of freedom is challenging. In this work, we use the molecular dihedral angles as features and explore the possibility of performing molecular conformer search in a latent space with a generative model named variational auto-encoder (VAE). We bias the VAE towards low-energy molecular configurations to generate more informative data. In this way, we can effectively build a reliable energy model for the low-energy potential energy surface. After the energy model has been built, we extract local-minimum conformations and refine them with structure optimization. We have tested and benchmarked our low-energy latent-space (LOLS) structure search method on organic molecules with $5-9$ searching dimensions. Our results agree with previous studies.

preprint2022arXiv

Multi-modal learning for predicting the genotype of glioma

The isocitrate dehydrogenase (IDH) gene mutation is an essential biomarker for the diagnosis and prognosis of glioma. It is promising to better predict glioma genotype by integrating focal tumor image and geometric features with brain network features derived from MRI. Convolutions neural networks show reasonable performance in predicting IDH mutation, which, however, cannot learn from non-Euclidean data, e.g., geometric and network data. In this study, we propose a multi-modal learning framework using three separate encoders to extract features of focal tumor image, tumor geometrics and global brain networks. To mitigate the limited availability of diffusion MRI, we develop a self-supervised approach to generate brain networks from anatomical multi-sequence MRI. Moreover, to extract tumor-related features from the brain network, we design a hierarchical attention module for the brain network encoder. Further, we design a bi-level multi-modal contrastive loss to align the multi-modal features and tackle the domain gap at the focal tumor and global brain. Finally, we propose a weighted population graph to integrate the multi-modal features for genotype prediction. Experimental results on the testing set show that the proposed model outperforms the baseline deep learning models. The ablation experiments validate the performance of different components of the framework. The visualized interpretation corresponds to clinical knowledge with further validation. In conclusion, the proposed learning framework provides a novel approach for predicting the genotype of glioma.

preprint2022arXiv

Nested sampling for physical scientists

We review Skilling's nested sampling (NS) algorithm for Bayesian inference and more broadly multi-dimensional integration. After recapitulating the principles of NS, we survey developments in implementing efficient NS algorithms in practice in high-dimensions, including methods for sampling from the so-called constrained prior. We outline the ways in which NS may be applied and describe the application of NS in three scientific fields in which the algorithm has proved to be useful: cosmology, gravitational-wave astronomy, and materials science. We close by making recommendations for best practice when using NS and by summarizing potential limitations and optimizations of NS.

preprint2022arXiv

Neural 3D Reconstruction in the Wild

We are witnessing an explosion of neural implicit representations in computer vision and graphics. Their applicability has recently expanded beyond tasks such as shape generation and image-based rendering to the fundamental problem of image-based 3D reconstruction. However, existing methods typically assume constrained 3D environments with constant illumination captured by a small set of roughly uniformly distributed cameras. We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections in the presence of varying illumination. To achieve this, we propose a hybrid voxel- and surface-guided sampling technique that allows for more efficient ray sampling around surfaces and leads to significant improvements in reconstruction quality. Further, we present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes. We perform extensive experiments, demonstrating that our approach surpasses both classical and neural reconstruction methods on a wide variety of metrics.

preprint2022arXiv

No Weighted-Regret Learning in Adversarial Bandits with Delays

Consider a scenario where a player chooses an action in each round $t$ out of $T$ rounds and observes the incurred cost after a delay of $d_{t}$ rounds. The cost functions and the delay sequence are chosen by an adversary. We show that in a non-cooperative game, the expected weighted ergodic distribution of play converges to the set of coarse correlated equilibria if players use algorithms that have "no weighted-regret" in the above scenario, even if they have linear regret due to too large delays. For a two-player zero-sum game, we show that no weighted-regret is sufficient for the weighted ergodic average of play to converge to the set of Nash equilibria. We prove that the FKM algorithm with $n$ dimensions achieves an expected regret of $O\left(nT^{\frac{3}{4}}+\sqrt{n}T^{\frac{1}{3}}D^{\frac{1}{3}}\right)$ and the EXP3 algorithm with $K$ arms achieves an expected regret of $O\left(\sqrt{\log K\left(KT+D\right)}\right)$ even when $D=\sum_{t=1}^{T}d_{t}$ and $T$ are unknown. These bounds use a novel doubling trick that, under mild assumptions, provably retains the regret bound for when $D$ and $T$ are known. Using these bounds, we show that FKM and EXP3 have no weighted-regret even for $d_{t}=O\left(t\log t\right)$. Therefore, algorithms with no weighted-regret can be used to approximate a CCE of a finite or convex unknown game that can only be simulated with bandit feedback, even if the simulation involves significant delays.

preprint2022arXiv

On Digital Subcarrier Multiplexing under A Bandwidth Limitation and ASE Noise

We show that digital subcarrier multiplexing (DSM) systems require much greater complexity for Nyquist pulse shaping than single-carrier (SC) systems, and it is a misconception that both systems use the same bandwidth when using the same pulse shaping. Through back-to-back (B2B) experiments with realistic transmitter (TX) modules and amplified spontaneous emission (ASE) noise loading, we show that even with optimized waterfilling and entropy loading, DSM does not achieve a larger net data rate (NDR) compared to SC when only ASE noise exists in the channel in long-haul transmission scenarios.

preprint2022arXiv

On the Complexity of Dynamic Submodular Maximization

We study dynamic algorithms for the problem of maximizing a monotone submodular function over a stream of $n$ insertions and deletions. We show that any algorithm that maintains a $(0.5+ε)$-approximate solution under a cardinality constraint, for any constant $ε>0$, must have an amortized query complexity that is $\mathit{polynomial}$ in $n$. Moreover, a linear amortized query complexity is needed in order to maintain a $0.584$-approximate solution. This is in sharp contrast with recent dynamic algorithms of [LMNF+20, Mon20] that achieve $(0.5-ε)$-approximation with a $\mathsf{poly}\log(n)$ amortized query complexity. On the positive side, when the stream is insertion-only, we present efficient algorithms for the problem under a cardinality constraint and under a matroid constraint with approximation guarantee $1-1/e-ε$ and amortized query complexities $\smash{O(\log (k/ε)/ε^2)}$ and $\smash{k^{\tilde{O}(1/ε^2)}\log n}$, respectively, where $k$ denotes the cardinality parameter or the rank of the matroid.

preprint2022arXiv

Optimization of Directional Landmark Deployment for Visual Observer on SE(3)

An optimization method is proposed in this paper for novel deployment of given number of directional landmarks (location and pose) within a given region in the 3-D task space. This new deployment technique is built on the geometric models of both landmarks and the monocular camera. In particular, a new concept of Multiple Coverage Probability (MCP) is defined to characterize the probability of at least n landmarks being covered simultaneously by a camera at a fixed position. The optimization is conducted with respect to the position and pose of the given number of landmarks to maximize MCP through globally exploration of the given 3-D space. By adopting the elimination genetic algorithm, the global optimal solutions can be obtained, which are then applied to improve the convergent performance of the visual observer on SE(3) as a demonstration example. Both simulation and experimental results are presented to validate the effectiveness of the proposed landmark deployment optimization method.

preprint2022arXiv

PAC-Bayes Information Bottleneck

Understanding the source of the superior generalization ability of NNs remains one of the most important problems in ML research. There have been a series of theoretical works trying to derive non-vacuous bounds for NNs. Recently, the compression of information stored in weights (IIW) is proved to play a key role in NNs generalization based on the PAC-Bayes theorem. However, no solution of IIW has ever been provided, which builds a barrier for further investigation of the IIW's property and its potential in practical deep learning. In this paper, we propose an algorithm for the efficient approximation of IIW. Then, we build an IIW-based information bottleneck on the trade-off between accuracy and information complexity of NNs, namely PIB. From PIB, we can empirically identify the fitting to compressing phase transition during NNs' training and the concrete connection between the IIW compression and the generalization. Besides, we verify that IIW is able to explain NNs in broad cases, e.g., varying batch sizes, over-parameterization, and noisy labels. Moreover, we propose an MCMC-based algorithm to sample from the optimal weight posterior characterized by PIB, which fulfills the potential of IIW in enhancing NNs in practice.

preprint2022arXiv

PACTran: PAC-Bayesian Metrics for Estimating the Transferability of Pretrained Models to Classification Tasks

With the increasing abundance of pretrained models in recent years, the problem of selecting the best pretrained checkpoint for a particular downstream classification task has been gaining increased attention. Although several methods have recently been proposed to tackle the selection problem (e.g. LEEP, H-score), these methods resort to applying heuristics that are not well motivated by learning theory. In this paper we present PACTran, a theoretically grounded family of metrics for pretrained model selection and transferability measurement. We first show how to derive PACTran metrics from the optimal PAC-Bayesian bound under the transfer learning setting. We then empirically evaluate three metric instantiations of PACTran on a number of vision tasks (VTAB) as well as a language-and-vision (OKVQA) task. An analysis of the results shows PACTran is a more consistent and effective transferability measure compared to existing selection methods.

preprint2022arXiv

Radio properties of the OH megamaser galaxy IIZw 096

Based on the two epochs EVN archive data from OH line observations of IIZw 096, we confirm that the high-resolution OH emission in this source mainly comes from two spots (OH1 and OH2) of comp D1 of this merging system. We found no significant variations in the OH line emission. The OH 1665 MHz line emission is detected at about 6 $σ$ level in the OH1 region by combining two epoch EVN observations. We found that the comp D1 shows the brightest CO, HCO+ line emission, as well as multi-band radio continuum emission. The environment around D1 shows no clear velocity structure associated with circular motions, making it different from most other OHMs in the literature, which might have been caused by an effect during the merger stage. Meanwhile, we found that the CO emission shows three velocity structures around D1, including the central broad FWHM region, the double peak region where the CO line profile shows two separated peaks, and the region of the high-velocity clouds where the CO line peaks at a high velocity ($\sim$ 11000 \kms). \HI in absorption also show high-velocity clouds around the D1 region, which might be due to inflows caused by the merging of two or more galaxy components. Based on the high-resolution K-band VLA and L-band VLBA observations of the radio continuum emission, we derived the brightness temperature in the range $10^{5}$ K to $10^{6}$ K, which is consistent with other starburst dominant OHM sources in the literature. The multi-band VLA observations show that the radio continuum emission of comp D might also have contributions from free-free emission, besides synchrotron emission. As a concenquence, these results support a starburst origin for the OHMs, without the presence of an AGN.

preprint2022arXiv

Robust Dynamic Assortment Optimization in the Presence of Outlier Customers

We consider the dynamic assortment optimization problem under the multinomial logit model (MNL) with unknown utility parameters. The main question investigated in this paper is model mis-specification under the $\varepsilon$-contamination model, which is a fundamental model in robust statistics and machine learning. In particular, throughout a selling horizon of length $T$, we assume that customers make purchases according to a well specified underlying multinomial logit choice model in a $(1-\varepsilon)$-fraction of the time periods, and make arbitrary purchasing decisions instead in the remaining $\varepsilon$-fraction of the time periods. In this model, we develop a new robust online assortment optimization policy via an active elimination strategy. We establish both upper and lower bounds on the regret, and show that our policy is optimal up to logarithmic factor in $T$ when the assortment capacity is constant. %% capacity of assortments has a constant upper limit. We further develop a fully adaptive policy that does not require any prior knowledge of the contamination parameter $\varepsilon$. In the case of the existence a sub-optimality gap between optimal and sub-optimal products, we also established gap-dependent logarithmic regret upper bounds and lower bounds in both the known-$\varepsilon$ and unknown-$\varepsilon$ cases. Our simulation study shows that our policy outperforms the existing policies based on upper confidence bounds (UCB) and Thompson sampling.

preprint2022arXiv

Robust One Round Federated Learning with Predictive Space Bayesian Inference

Making predictions robust is an important challenge. A separate challenge in federated learning (FL) is to reduce the number of communication rounds, particularly since doing so reduces performance in heterogeneous data settings. To tackle both issues, we take a Bayesian perspective on the problem of learning a global model. We show how the global predictive posterior can be approximated using client predictive posteriors. This is unlike other works which aggregate the local model space posteriors into the global model space posterior, and are susceptible to high approximation errors due to the posterior's high dimensional multimodal nature. In contrast, our method performs the aggregation on the predictive posteriors, which are typically easier to approximate owing to the low-dimensionality of the output space. We present an algorithm based on this idea, which performs MCMC sampling at each client to obtain an estimate of the local posterior, and then aggregates these in one round to obtain a global ensemble model. Through empirical evaluation on several classification and regression tasks, we show that despite using one round of communication, the method is competitive with other FL techniques, and outperforms them on heterogeneous settings. The code is publicly available at https://github.com/hasanmohsin/FedPredSpace_1Round.

preprint2022arXiv

S2MS: Self-Supervised Learning Driven Multi-Spectral CT Image Enhancement

Photon counting spectral CT (PCCT) can produce reconstructed attenuation maps in different energy channels, reflecting energy properties of the scanned object. Due to the limited photon numbers and the non-ideal detector response of each energy channel, the reconstructed images usually contain much noise. With the development of Deep Learning (DL) technique, different kinds of DL-based models have been proposed for noise reduction. However, most of the models require clean data set as the training labels, which are not always available in medical imaging field. Inspiring by the similarities of each channel's reconstructed image, we proposed a self-supervised learning based PCCT image enhancement framework via multi-spectral channels (S2MS). In S2MS framework, both the input and output labels are noisy images. Specifically, one single channel image was used as output while images of other single channels and channel-sum image were used as input to train the network, which can fully use the spectral data information without extra cost. The simulation results based on the AAPM Low-dose CT Challenge database showed that the proposed S2MS model can suppress the noise and preserve details more effectively in comparison with the traditional DL models, which has potential to improve the image quality of PCCT in clinical applications.

preprint2022arXiv

Shortcuts to Adiabaticity for Fast Qubit Readout in Circuit Quantum Electrodynamics

We propose how to engineer the longitudinal coupling to accelerate the measurement of a qubit longitudinally coupled to a cavity, motivated by the concept of shortcuts to adiabaticity. Different modulations are inversely designed from two methods of inverse engineering and counter-diabatic driving, for achieving larger values of the signal-to-noise ratio (SNR) at nanosecond scale. By comparison, we demonstrate that our protocols outperform the usual periodic modulations on the pointer state separation and SNR. Finally, we show a possible implementation considering state-of-the-art circuit quantum electrodynamics architecture, estimating the minimal time allowed for the measurement process.

preprint2022arXiv

Two-stage Hypothesis Tests for Variable Interactions with FDR Control

In many scenarios such as genome-wide association studies where dependences between variables commonly exist, it is often of interest to infer the interaction effects in the model. However, testing pairwise interactions among millions of variables in complex and high-dimensional data suffers from low statistical power and huge computational cost. To address these challenges, we propose a two-stage testing procedure with false discovery rate (FDR) control, which is known as a less conservative multiple-testing correction. Theoretically, the difficulty in the FDR control dues to the data dependence among test statistics in two stages, and the fact that the number of hypothesis tests conducted in the second stage depends on the screening result in the first stage. By using the Cramér type moderate deviation technique, we show that our procedure controls FDR at the desired level asymptotically in the generalized linear model (GLM), where the model is allowed to be misspecified. In addition, the asymptotic power of the FDR control procedure is rigorously established. We demonstrate via comprehensive simulation studies that our two-stage procedure is computationally more efficient than the classical BH procedure, with a comparable or improved statistical power. Finally, we apply the proposed method to a bladder cancer data from dbGaP where the scientific goal is to identify genetic susceptibility loci for bladder cancer.

preprint2022arXiv

Unbiased Implicit Feedback via Bi-level Optimization

Implicit feedback is widely leveraged in recommender systems since it is easy to collect and provides weak supervision signals. Recent works reveal a huge gap between the implicit feedback and user-item relevance due to the fact that implicit feedback is also closely related to the item exposure. To bridge this gap, existing approaches explicitly model the exposure and propose unbiased estimators to improve the relevance. Unfortunately, these unbiased estimators suffer from the high gradient variance, especially for long-tail items, leading to inaccurate gradient updates and degraded model performance. To tackle this challenge, we propose a low-variance unbiased estimator from a probabilistic perspective, which effectively bounds the variance of the gradient. Unlike previous works which either estimate the exposure via heuristic-based strategies or use a large biased training set, we propose to estimate the exposure via an unbiased small-scale validation set. Specifically, we first parameterize the user-item exposure by incorporating both user and item information, and then construct an unbiased validation set from the biased training set. By leveraging the unbiased validation set, we adopt bi-level optimization to automatically update exposure-related parameters along with recommendation model parameters during the learning. Experiments on two real-world datasets and two semi-synthetic datasets verify the effectiveness of our method.

preprint2021arXiv

Adversarial Combinatorial Bandits with General Non-linear Reward Functions

In this paper we study the adversarial combinatorial bandit with a known non-linear reward function, extending existing work on adversarial linear combinatorial bandit. {The adversarial combinatorial bandit with general non-linear reward is an important open problem in bandit literature, and it is still unclear whether there is a significant gap from the case of linear reward, stochastic bandit, or semi-bandit feedback.} We show that, with $N$ arms and subsets of $K$ arms being chosen at each of $T$ time periods, the minimax optimal regret is $\widetildeΘ_{d}(\sqrt{N^d T})$ if the reward function is a $d$-degree polynomial with $d< K$, and $Θ_K(\sqrt{N^K T})$ if the reward function is not a low-degree polynomial. {Both bounds are significantly different from the bound $O(\sqrt{\mathrm{poly}(N,K)T})$ for the linear case, which suggests that there is a fundamental gap between the linear and non-linear reward structures.} Our result also finds applications to adversarial assortment optimization problem in online recommendation. We show that in the worst-case of adversarial assortment problem, the optimal algorithm must treat each individual $\binom{N}{K}$ assortment as independent.

preprint2021arXiv

Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms

Reinforcement learning methods can achieve significant performance but require a large amount of training data collected on the same robotic platform. A policy trained with expensive data is rendered useless after making even a minor change to the robot hardware. In this paper, we address the challenging problem of adapting a policy, trained to perform a task, to a novel robotic hardware platform given only few demonstrations of robot motion trajectories on the target robot. We formulate it as a few-shot meta-learning problem where the goal is to find a meta-model that captures the common structure shared across different robotic platforms such that data-efficient adaptation can be performed. We achieve such adaptation by introducing a learning framework consisting of a probabilistic gradient-based meta-learning algorithm that models the uncertainty arising from the few-shot setting with a low-dimensional latent variable. We experimentally evaluate our framework on a simulated reaching and a real-robot picking task using 400 simulated robots generated by varying the physical parameters of an existing set of robotic platforms. Our results show that the proposed method can successfully adapt a trained policy to different robotic platforms with novel physical parameters and the superiority of our meta-learning algorithm compared to state-of-the-art methods for the introduced few-shot policy adaptation problem.

preprint2021arXiv

Confronting the Carbon-footprint Challenge of Blockchain

The distributed consensus mechanism is the backbone of the rapidly developing blockchain network. Blockchain platforms consume vast amounts of electricity based on the current consensus mechanism of Proof of Work. Here, we point out an advanced consensus mechanism named Proof of Stake that can eliminate the extensive energy consumption of the current PoW-based blockchain. We comprehensively elucidate the current and projected energy consumption and carbon footprint of the PoW and PoS based Bitcoin and Ethereum blockchain platforms.

preprint2021arXiv

Connecting The Dots To Combat Collective Fraud

Modern fraudsters write malicious programs to coordinate a group of accounts to commit collective fraud for illegal profits in online platforms. These programs have access to a set of finite resources - a set of IPs, devices, and accounts etc. and sometime manipulate fake accounts to collaboratively attack the target system. Inspired by these observations, we share our experience in building two real-time risk control systems to detect collective fraud. We show that with TigerGraph, a powerful graph database, and its innovative query language - GSQL, data scientists and fraud experts can conveniently implement and deploy an end-to-end risk control system as a graph database application.

preprint2021arXiv

Connection between inverse engineering and optimal control in shortcuts to adiabaticity

We consider fast high-fidelity quantum control by using a shortcut to adiabaticity (STA) technique and optimal control theory (OCT). Three specific examples, including expansion of cold atoms from the harmonic trap, atomic transport by moving harmonic trap, and spin dynamics in the presence of dissipation, are explicitly detailed. Using OCT as a qualitative guide, we demonstrate how STA protocols designed from inverse engineering method, can approach with very high precision optimal solutions built about physical constraints, by a proper choice of the interpolation function and with a very reduced number of adjustable parameters.

preprint2021arXiv

Dimension Independent Generalization Error by Stochastic Gradient Descent

One classical canon of statistics is that large models are prone to overfitting, and model selection procedures are necessary for high dimensional data. However, many overparameterized models, such as neural networks, perform very well in practice, although they are often trained with simple online methods and regularization. The empirical success of overparameterized models, which is often known as benign overfitting, motivates us to have a new look at the statistical generalization theory for online optimization. In particular, we present a general theory on the generalization error of stochastic gradient descent (SGD) solutions for both convex and locally convex loss functions. We further discuss data and model conditions that lead to a ``low effective dimension". Under these conditions, we show that the generalization error either does not depend on the ambient dimension $p$ or depends on $p$ via a poly-logarithmic factor. We also demonstrate that in several widely used statistical models, the ``low effective dimension'' arises naturally in overparameterized settings. The studied statistical applications include both convex models such as linear regression and logistic regression and non-convex models such as $M$-estimator and two-layer neural networks.

preprint2021arXiv

Distributed Estimation for Principal Component Analysis: an Enlarged Eigenspace Analysis

The growing size of modern data sets brings many challenges to the existing statistical estimation approaches, which calls for new distributed methodologies. This paper studies distributed estimation for a fundamental statistical machine learning problem, principal component analysis (PCA). Despite the massive literature on top eigenvector estimation, much less is presented for the top-$L$-dim ($L>1$) eigenspace estimation, especially in a distributed manner. We propose a novel multi-round algorithm for constructing top-$L$-dim eigenspace for distributed data. Our algorithm takes advantage of shift-and-invert preconditioning and convex optimization. Our estimator is communication-efficient and achieves a fast convergence rate. In contrast to the existing divide-and-conquer algorithm, our approach has no restriction on the number of machines. Theoretically, the traditional Davis-Kahan theorem requires the explicit eigengap assumption to estimate the top-$L$-dim eigenspace. To abandon this eigengap assumption, we consider a new route in our analysis: instead of exactly identifying the top-$L$-dim eigenspace, we show that our estimator is able to cover the targeted top-$L$-dim population eigenspace. Our distributed algorithm can be applied to a wide range of statistical problems based on PCA, such as principal component regression and single index model. Finally, We provide simulation studies to demonstrate the performance of the proposed distributed estimator.

preprint2021arXiv

DoubleEnsemble: A New Ensemble Method Based on Sample Reweighting and Feature Selection for Financial Data Analysis

Modern machine learning models (such as deep neural networks and boosting decision tree models) have become increasingly popular in financial market prediction, due to their superior capacity to extract complex non-linear patterns. However, since financial datasets have very low signal-to-noise ratio and are non-stationary, complex models are often very prone to overfitting and suffer from instability issues. Moreover, as various machine learning and data mining tools become more widely used in quantitative trading, many trading firms have been producing an increasing number of features (aka factors). Therefore, how to automatically select effective features becomes an imminent problem. To address these issues, we propose DoubleEnsemble, an ensemble framework leveraging learning trajectory based sample reweighting and shuffling based feature selection. Specifically, we identify the key samples based on the training dynamics on each sample and elicit key features based on the ablation impact of each feature via shuffling. Our model is applicable to a wide range of base models, capable of extracting complex patterns, while mitigating the overfitting and instability issues for financial market prediction. We conduct extensive experiments, including price prediction for cryptocurrencies and stock trading, using both DNN and gradient boosting decision tree as base models. Our experiment results demonstrate that DoubleEnsemble achieves a superior performance compared with several baseline methods.

preprint2021arXiv

Dynamic Assortment Selection under the Nested Logit Models

We study a stylized dynamic assortment planning problem during a selling season of finite length $T$. At each time period, the seller offers an arriving customer an assortment of substitutable products and the customer makes the purchase among offered products according to a discrete choice model. The goal of the seller is to maximize the expected revenue, or equivalently, to minimize the worst-case expected regret. One key challenge is that utilities of products are unknown to the seller and need to be learned. Although the dynamic assortment planning problem has received increasing attention in revenue management, most existing work is based on the multinomial logit choice models (MNL). In this paper, we study the problem of dynamic assortment planning under a more general choice model -- the nested logit model, which models hierarchical choice behavior and is ``the most widely used member of the GEV (generalized extreme value) family''. By leveraging the revenue-ordered structure of the optimal assortment within each nest, we develop a novel upper confidence bound (UCB) policy with an aggregated estimation scheme. Our policy simultaneously learns customers' choice behavior and makes dynamic decisions on assortments based on the current knowledge. It achieves the accumulated regret at the order of $\tilde{O}(\sqrt{MNT})$, where $M$ is the number of nests and $N$ is the number of products in each nest. We further provide a lower bound result of $Ω(\sqrt{MT})$, which shows the near optimality of the upper bound when $T$ is much larger than $M$ and $N$. When the number of items per nest $N$ is large, we further provide a discretization heuristic for better performance of our algorithm. Numerical results are presented to demonstrate the empirical performance of our proposed algorithms.

preprint2021arXiv

Echo state graph neural networks with analogue random resistor arrays

Recent years have witnessed an unprecedented surge of interest, from social networks to drug discovery, in learning representations of graph-structured data. However, graph neural networks, the machine learning models for handling graph-structured data, face significant challenges when running on conventional digital hardware, including von Neumann bottleneck incurred by physically separated memory and processing units, slowdown of Moore's law due to transistor scaling limit, and expensive training cost. Here we present a novel hardware-software co-design, the random resistor array-based echo state graph neural network, which addresses these challenges. The random resistor arrays not only harness low-cost, nanoscale and stackable resistors for highly efficient in-memory computing using simple physical laws, but also leverage the intrinsic stochasticity of dielectric breakdown to implement random projections in hardware for an echo state network that effectively minimizes the training cost thanks to its fixed and random weights. The system demonstrates state-of-the-art performance on both graph classification using the MUTAG and COLLAB datasets and node classification using the CORA dataset, achieving 34.2x, 93.2x, and 570.4x improvement of energy efficiency and 98.27%, 99.46%, and 95.12% reduction of training cost compared to conventional graph learning on digital hardware, respectively, which may pave the way for the next generation AI system for graph learning.

preprint2021arXiv

Enquire One's Parent and Child Before Decision: Fully Exploit Hierarchical Structure for Self-Supervised Taxonomy Expansion

Taxonomy is a hierarchically structured knowledge graph that plays a crucial role in machine intelligence. The taxonomy expansion task aims to find a position for a new term in an existing taxonomy to capture the emerging knowledge in the world and keep the taxonomy dynamically updated. Previous taxonomy expansion solutions neglect valuable information brought by the hierarchical structure and evaluate the correctness of merely an added edge, which downgrade the problem to node-pair scoring or mini-path classification. In this paper, we propose the Hierarchy Expansion Framework (HEF), which fully exploits the hierarchical structure's properties to maximize the coherence of expanded taxonomy. HEF makes use of taxonomy's hierarchical structure in multiple aspects: i) HEF utilizes subtrees containing most relevant nodes as self-supervision data for a complete comparison of parental and sibling relations; ii) HEF adopts a coherence modeling module to evaluate the coherence of a taxonomy's subtree by integrating hypernymy relation detection and several tree-exclusive features; iii) HEF introduces the Fitting Score for position selection, which explicitly evaluates both path and level selections and takes full advantage of parental relations to interchange information for disambiguation and self-correction. Extensive experiments show that by better exploiting the hierarchical structure and optimizing taxonomy's coherence, HEF vastly surpasses the prior state-of-the-art on three benchmark datasets by an average improvement of 46.7% in accuracy and 32.3% in mean reciprocal rank.

preprint2021arXiv

Experimental quantum simulation of superradiant phase transition beyond no-go theorem via antisqueezing

Superradiant phase transition (SPT) in thermal equilibrium, as a fundamental concept bridging the statistical physics and electrodynamics, can offer the key resources for quantum information science. Notwithstanding its fundamental and practical significances, equilibrium SPT has never been observed in experiments since the first proposal in the 1970s. Furthermore, the existence of equilibrium SPT in the cavity quantum electrodynamics (QED) systems is still subject of ongoing debates, due to the no-go theorem induced by the so-called A2 term. Based on the platform of nuclear magnetic resonance (NMR), here we experimentally demonstrate the occurrence of equilibrium SPT beyond no-go theorem by introducing the antisqueezing effect. The mechanism relies on the antisqueezing that recovers the singularity of the ground state via exponentially enhancing the zero point fluctuation (ZPF) of system. The strong entanglement and squeezed Schrodinger cat states of spins are achieved experimentally in the superradiant phase, which may play an important role in fundamental tests of quantum theory, implementing quantum metrology and high-efficient quantum information processing. Our experiment also shows the antisqueezing-enhanced signal-to-noise rate (SNR) of NMR spectrum, providing a general method for implementing high-precision NMR experiments.

preprint2021arXiv

Experimentally Realizing Efficient Quantum Control with Reinforcement Learning

Robust and high-precision quantum control is crucial but challenging for scalable quantum computation and quantum information processing. Traditional adiabatic control suffers severe limitations on gate performance imposed by environmentally induced noise because of a quantum system's limited coherence time. In this work, we experimentally demonstrate an alternative approach {to quantum control} based on deep reinforcement learning (DRL) on a trapped $^{171}\mathrm{Yb}^{+}$ ion. In particular, we find that DRL leads to fast and robust {digital quantum operations with running time bounded by shortcuts to adiabaticity} (STA). Besides, we demonstrate that DRL's robustness against both Rabi and detuning errors can be achieved simultaneously without any input from STA. Our experiments reveal a general framework of digital quantum control, leading to a promising enhancement in quantum information processing.

preprint2021arXiv

Fast-forward scaling of atom-molecule conversion in Bose-Einstein condensates

Robust stimulated Raman exact passages are requisite for controlling nonlinear quantum systems, with the wide applications ranging from ultracold molecules, non-linear optics to superchemistry. Inspired by shortcuts to adiabaticity, we propose the fast-forward scaling of stimulated Raman adiabatic processes with the nonlinearity involved, describing the transfer from an atomic Bose-Einstein condensate to a molecular one by controllable external fields. The fidelity and robustness of atom-molecule conversion are shown to surpass those of conventional adiabatic passages, assisted by fast-forward driving field. Finally, our results are extended to the fractional stimulated Raman adiabatic processes for the coherent superposition of atomic and molecular states.

preprint2021arXiv

First-order Newton-type Estimator for Distributed Estimation and Inference

This paper studies distributed estimation and inference for a general statistical problem with a convex loss that could be non-differentiable. For the purpose of efficient computation, we restrict ourselves to stochastic first-order optimization, which enjoys low per-iteration complexity. To motivate the proposed method, we first investigate the theoretical properties of a straightforward Divide-and-Conquer Stochastic Gradient Descent (DC-SGD) approach. Our theory shows that there is a restriction on the number of machines and this restriction becomes more stringent when the dimension $p$ is large. To overcome this limitation, this paper proposes a new multi-round distributed estimation procedure that approximates the Newton step only using stochastic subgradient. The key component in our method is the proposal of a computationally efficient estimator of $Σ^{-1} w$, where $Σ$ is the population Hessian matrix and $w$ is any given vector. Instead of estimating $Σ$ (or $Σ^{-1}$) that usually requires the second-order differentiability of the loss, the proposed First-Order Newton-type Estimator (FONE) directly estimates the vector of interest $Σ^{-1} w$ as a whole and is applicable to non-differentiable losses. Our estimator also facilitates the inference for the empirical risk minimizer. It turns out that the key term in the limiting covariance has the form of $Σ^{-1} w$, which can be estimated by FONE.

preprint2021arXiv

Hybrid quantum-classical approach to enhanced quantum metrology

Quantum metrology plays a fundamental role in many scientific areas. However, the complexity of engineering entangled probes and the external noise raise technological barriers for realizing the expected precision of the to-be-estimated parameter with given resources. Here, we address this problem by introducing adjustable controls into the encoding process and then utilizing a hybrid quantum-classical approach to automatically optimize the controls online. Our scheme does not require any complex or intractable off-line design, and it can inherently correct certain unitary errors during the learning procedure. We also report the first experimental demonstration of this promising scheme for the task of finding optimal probes for frequency estimation on a nuclear magnetic resonance (NMR) processor. The proposed scheme paves the way to experimentally auto-search optimal protocol for improving the metrology precision.

preprint2021arXiv

Implementation of a Hybrid Classical-Quantum Annealing Algorithm for Logistic Network Design

The logistic network design is an abstract optimization problem that, under the assumption of minimal cost, seeks the optimal configuration of the supply chain's infrastructures and facilities based on customer demand. Key economic decisions are taken about the location, number, and size of manufacturing facilities and warehouses based on the optimal solution. Therefore, improvements in the methods to address this question, which is known to be in the NP-hard complexity class, would have relevant financial consequences. Here, we implement in the D-Wave quantum annealer a hybrid classical-quantum annealing algorithm. The cost function with constraints is translated to a spin Hamiltonian, whose ground state encodes the searched result. As a benchmark, we measure the accuracy of results for a set of paradigmatic problems against the optimal published solutions (the error is on average below $1\%$), and the performance is compared against the classical algorithm, showing a remarkable reduction in the number of iterations. This work shows that state-of-the-art quantum annealers may codify and solve relevant supply-chain problems even still far from useful quantum supremacy.

preprint2021arXiv

Learning a Product Relevance Model from Click-Through Data in E-Commerce

The search engine plays a fundamental role in online e-commerce systems, to help users find the products they want from the massive product collections. Relevance is an essential requirement for e-commerce search, since showing products that do not match search query intent will degrade user experience. With the existence of vocabulary gap between user language of queries and seller language of products, measuring semantic relevance is necessary and neural networks are engaged to address this task. However, semantic relevance is different from click-through rate prediction in that no direct training signal is available. Most previous attempts learn relevance models from user click-through data that are cheap and abundant. Unfortunately, click behavior is noisy and misleading, which is affected by not only relevance but also factors including price, image and attractive titles. Therefore, it is challenging but valuable to learn relevance models from click-through data. In this paper, we propose a new relevance learning framework that concentrates on how to train a relevance model from the weak supervision of click-through data. Different from previous efforts that treat samples as either relevant or irrelevant, we construct more fine-grained samples for training. We propose a novel way to consider samples of different relevance confidence, and come up with a new training objective to learn a robust relevance model with desirable score distribution. The proposed model is evaluated on offline annotated data and online A/B testing, and it achieves both promising performance and high computational efficiency. The model has already been deployed online, serving the search traffic of Taobao for over a year.

preprint2021arXiv

Lifelong Learning based Disease Diagnosis on Clinical Notes

Current deep learning based disease diagnosis systems usually fall short in catastrophic forgetting, i.e., directly fine-tuning the disease diagnosis model on new tasks usually leads to abrupt decay of performance on previous tasks. What is worse, the trained diagnosis system would be fixed once deployed but collecting training data that covers enough diseases is infeasible, which inspires us to develop a lifelong learning diagnosis system. In this work, we propose to adopt attention to combine medical entities and context, embedding episodic memory and consolidation to retain knowledge, such that the learned model is capable of adapting to sequential disease-diagnosis tasks. Moreover, we establish a new benchmark, named Jarvis-40, which contains clinical notes collected from various hospitals. Our experiments show that the proposed method can achieve state-of-the-art performance on the proposed benchmark.

preprint2021arXiv

Online Disease Self-diagnosis with Inductive Heterogeneous Graph Convolutional Networks

We propose a Healthcare Graph Convolutional Network (HealGCN) to offer disease self-diagnosis service for online users based on Electronic Healthcare Records (EHRs). Two main challenges are focused in this paper for online disease diagnosis: (1) serving cold-start users via graph convolutional networks and (2) handling scarce clinical description via a symptom retrieval system. To this end, we first organize the EHR data into a heterogeneous graph that is capable of modeling complex interactions among users, symptoms and diseases, and tailor the graph representation learning towards disease diagnosis with an inductive learning paradigm. Then, we build a disease self-diagnosis system with a corresponding EHR Graph-based Symptom Retrieval System (GraphRet) that can search and provide a list of relevant alternative symptoms by tracing the predefined meta-paths. GraphRet helps enrich the seed symptom set through the EHR graph when confronting users with scarce descriptions, hence yield better diagnosis accuracy. At last, we validate the superiority of our model on a large-scale EHR dataset.

preprint2021arXiv

Random Restrictions of High-Dimensional Distributions and Uniformity Testing with Subcube Conditioning

We give a nearly-optimal algorithm for testing uniformity of distributions supported on $\{-1,1\}^n$, which makes $\tilde O (\sqrt{n}/\varepsilon^2)$ queries to a subcube conditional sampling oracle (Bhattacharyya and Chakraborty (2018)). The key technical component is a natural notion of random restriction for distributions on $\{-1,1\}^n$, and a quantitative analysis of how such a restriction affects the mean vector of the distribution. Along the way, we consider the problem of mean testing with independent samples and provide a nearly-optimal algorithm.

preprint2021arXiv

Shape-Enforcing Operators for Point and Interval Estimators

A common problem in econometrics, statistics, and machine learning is to estimate and make inference on functions that satisfy shape restrictions. For example, distribution functions are nondecreasing and range between zero and one, height growth charts are nondecreasing in age, and production functions are nondecreasing and quasi-concave in input quantities. We propose a method to enforce these restrictions ex post on point and interval estimates of the target function by applying functional operators. If an operator satisfies certain properties that we make precise, the shape-enforced point estimates are closer to the target function than the original point estimates and the shape-enforced interval estimates have greater coverage and shorter length than the original interval estimates. We show that these properties hold for six different operators that cover commonly used shape restrictions in practice: range, convexity, monotonicity, monotone convexity, quasi-convexity, and monotone quasi-convexity. We illustrate the results with two empirical applications to the estimation of a height growth chart for infants in India and a production function for chemical firms in China.

preprint2021arXiv

Some continuous and discontinuous Galerkin methods and structure preservation for incompressible flows

In this paper, we present consistent and inconsistent discontinuous Galerkin methods for incompressible Euler and Navier-Stokes equations with the kinematic pressure, Bernoulli function and EMAC function. Semi- and fully discrete energy stability of the proposed dG methods are proved in a unified fashion. Conservation of total energy, linear and angular momentum is discussed with both central and upwind fluxes. Numerical experiments are presented to demonstrate our findings and compare our schemes with conventional schemes in the literature in both unsteady and steady problems. Numerical results show that global conservation of the physical quantities may not be enough to demonstrate the performance of the schemes, and our schemes are competitive and able to capture essential physical features in several benchmark problems.

preprint2021arXiv

Tight Regret Bounds for Infinite-armed Linear Contextual Bandits

Linear contextual bandit is an important class of sequential decision making problems with a wide range of applications to recommender systems, online advertising, healthcare, and many other machine learning related tasks. While there is a lot of prior research, tight regret bounds of linear contextual bandit with infinite action sets remain open. In this paper, we address this open problem by considering the linear contextual bandit with (changing) infinite action sets. We prove a regret upper bound on the order of $O(\sqrt{d^2T\log T})\times \text{poly}(\log\log T)$ where $d$ is the domain dimension and $T$ is the time horizon. Our upper bound matches the previous lower bound of $Ω(\sqrt{d^2 T\log T})$ in [Li et al., 2019] up to iterated logarithmic terms.

preprint2021arXiv

Variance Reduced Median-of-Means Estimator for Byzantine-Robust Distributed Inference

This paper develops an efficient distributed inference algorithm, which is robust against a moderate fraction of Byzantine nodes, namely arbitrary and possibly adversarial machines in a distributed learning system. In robust statistics, the median-of-means (MOM) has been a popular approach to hedge against Byzantine failures due to its ease of implementation and computational efficiency. However, the MOM estimator has the shortcoming in terms of statistical efficiency. The first main contribution of the paper is to propose a variance reduced median-of-means (VRMOM) estimator, which improves the statistical efficiency over the vanilla MOM estimator and is computationally as efficient as the MOM. Based on the proposed VRMOM estimator, we develop a general distributed inference algorithm that is robust against Byzantine failures. Theoretically, our distributed algorithm achieves a fast convergence rate with only a constant number of rounds of communications. We also provide the asymptotic normality result for the purpose of statistical inference. To the best of our knowledge, this is the first normality result in the setting of Byzantine-robust distributed learning. The simulation results are also presented to illustrate the effectiveness of our method.

Xi Chen

Quick read

Decide how to stay connected

How to connect with this researcher

Open a focused conversation when the fit is right

See the researcher in context

Building this graph slice

132 published item(s)

PanoWorld: Towards Spatial Supersensing in 360$^\circ$ Panorama World

Demonstration of Hardware Efficient Photonic Variational Quantum Algorithm

OmniVCus: Feedforward Subject-driven Video Customization with Multimodal Control Conditions

Depth Map Denoising Network and Lightweight Fusion Network for Enhanced 3D Face Recognition

Transfer the linguistic representations from TTS to accent conversion with non-parallel data

A Lightweight Clustering Framework for Unsupervised Semantic Segmentation

High precision atom interferometer-based dynamic gravimeter measurement by eliminating the cross-coupling effect

LaFFi: Leveraging Hybrid Natural Language Feedback for Fine-tuning Language Models

Shortcuts to Adiabatic Soliton Compression in Active Nonlinear Kerr Media

Accelerating Adaptive Cubic Regularization of Newton&#39;s Method via Random Sampling

Active Learning for Contextual Search with Binary Feedbacks

All You May Need for VQA are Image Captions

AutoMO-Mixer: An automated multi-objective Mixer model for balanced, safe and robust prediction in medicine

Bayesian posterior repartitioning for nested sampling

Collaborative learning of images and geometrics for predicting isocitrate dehydrogenase status of glioma

Context-Based Dynamic Pricing with Online Clustering

Coverage Optimization of Camera Network for Continuous Deformable Object

Development of a compact high-resolution absolute gravity gradiometer based on atom interferometers

Digitized-counterdiabatic quantum approximate optimization algorithm

Digitized-Counterdiabatic Quantum Optimization

Dynamic Car Dispatching and Pricing: Revenue and Fairness for Ridesharing Platforms

Efficient broadband frequency conversion via shortcut to adiabaticity

Encoder-Decoder Architecture for Supervised Dynamic Graph Learning: A Survey

Finding Influential Instances for Distantly Supervised Relation Extraction

Fixed-Support Wasserstein Barycenters: Computational Hardness and Fast Algorithm

FocalClick: Towards Practical Interactive Image Segmentation

Improved Upper Bounds for Finding Tarski Fixed Points

Investigation of Singing Voice Separation for Singing Voice Detection in Polyphonic Music

Latent-Variable Advantage-Weighted Policy Optimization for Offline RL

Leveraging Intrinsic Gradient Information for Further Training of Differentiable Machine Learning Models

Machine-learning assisted quantum control in random environment

Memory Bounds for Continual Learning

Meta-Learning Digitized-Counterdiabatic Quantum Optimization

Mitigating Data Heterogeneity in Federated Learning with Data Augmentation

Molecular conformer search with low-energy latent space

Multi-modal learning for predicting the genotype of glioma

Nested sampling for physical scientists

Neural 3D Reconstruction in the Wild

No Weighted-Regret Learning in Adversarial Bandits with Delays

On Digital Subcarrier Multiplexing under A Bandwidth Limitation and ASE Noise

On the Complexity of Dynamic Submodular Maximization

Optimization of Directional Landmark Deployment for Visual Observer on SE(3)

PAC-Bayes Information Bottleneck

PACTran: PAC-Bayesian Metrics for Estimating the Transferability of Pretrained Models to Classification Tasks

Radio properties of the OH megamaser galaxy IIZw 096

Robust Dynamic Assortment Optimization in the Presence of Outlier Customers

Robust One Round Federated Learning with Predictive Space Bayesian Inference

S2MS: Self-Supervised Learning Driven Multi-Spectral CT Image Enhancement

Shortcuts to Adiabaticity for Fast Qubit Readout in Circuit Quantum Electrodynamics

Two-stage Hypothesis Tests for Variable Interactions with FDR Control

Unbiased Implicit Feedback via Bi-level Optimization

Adversarial Combinatorial Bandits with General Non-linear Reward Functions

Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platforms

Confronting the Carbon-footprint Challenge of Blockchain

Connecting The Dots To Combat Collective Fraud

Connection between inverse engineering and optimal control in shortcuts to adiabaticity

Dimension Independent Generalization Error by Stochastic Gradient Descent

Distributed Estimation for Principal Component Analysis: an Enlarged Eigenspace Analysis

DoubleEnsemble: A New Ensemble Method Based on Sample Reweighting and Feature Selection for Financial Data Analysis

Dynamic Assortment Selection under the Nested Logit Models

Echo state graph neural networks with analogue random resistor arrays

Enquire One&#39;s Parent and Child Before Decision: Fully Exploit Hierarchical Structure for Self-Supervised Taxonomy Expansion

Experimental quantum simulation of superradiant phase transition beyond no-go theorem via antisqueezing

Experimentally Realizing Efficient Quantum Control with Reinforcement Learning

Fast-forward scaling of atom-molecule conversion in Bose-Einstein condensates

First-order Newton-type Estimator for Distributed Estimation and Inference

Hybrid quantum-classical approach to enhanced quantum metrology

Implementation of a Hybrid Classical-Quantum Annealing Algorithm for Logistic Network Design

Learning a Product Relevance Model from Click-Through Data in E-Commerce

Lifelong Learning based Disease Diagnosis on Clinical Notes

Online Disease Self-diagnosis with Inductive Heterogeneous Graph Convolutional Networks

Random Restrictions of High-Dimensional Distributions and Uniformity Testing with Subcube Conditioning

Accelerating Adaptive Cubic Regularization of Newton's Method via Random Sampling

Enquire One's Parent and Child Before Decision: Fully Exploit Hierarchical Structure for Self-Supervised Taxonomy Expansion

Formation of a fast "lane" for positional transition in a microparticle-suspended nematic liquid crystal cell

Reply to "Comment on 'Apical charge flux-modulated in-plane transport properties of cuprate superconductors'"

Zariski's conjecture and Euler-Chow series