Source author record

Gary Cheng

Explore this author’s publications, collaborators and research topics. This catalog record has not been claimed by its author.

ResearcherUnclaimed source record

Machine Learning Computer Vision math.OC Computation and Language cond-mat.mes-hall cond-mat.mtrl-sci Distributed, Parallel, and Cluster Computing Information Retrieval

Catalog footprint

Research overview

8works

8topics

4close collaborators

Actions

Is this your work?

Open graph Browse papers

preprint2022arXiv

Approximate Function Evaluation via Multi-Armed Bandits

We study the problem of estimating the value of a known smooth function $f$ at an unknown point $\boldsymbolμ \in \mathbb{R}^n$ , where each component $μ_i$ can be sampled via a noisy oracle. Sampling more frequently components of $\boldsymbolμ$ corresponding to directions of the function with larger directional derivatives is more sample-efficient. However, as $\boldsymbolμ$ is unknown, the optimal sampling frequencies are also unknown. We design an instance-adaptive algorithm that learns to sample according to the importance of each coordinate, and with probability at least $1-δ$ returns an $ε$ accurate estimate of $f(\boldsymbolμ)$ . We generalize our algorithm to adapt to heteroskedastic noise, and prove asymptotic optimality when $f$ is linear. We corroborate our theoretical results with numerical experiments, showing the dramatic gains afforded by adaptivity.

preprint2022arXiv

Federated Asymptotics: a model to compare federated learning algorithms

We propose an asymptotic framework to analyze the performance of (personalized) federated learning algorithms. In this new framework, we formulate federated learning as a multi-criterion objective, where the goal is to minimize each client's loss using information from all of the clients. We analyze a linear regression model where, for a given client, we may theoretically compare the performance of various algorithms in the high-dimensional asymptotic limit. This asymptotic multi-criterion approach naturally models the high-dimensional, many-device nature of federated learning. These tools make fairly precise predictions about the benefits of personalization and information sharing in federated scenarios -- at least in our (stylized) model -- including that Federated Averaging with simple client fine-tuning achieves the same asymptotic risk as the more intricate meta-learning and proximal-regularized approaches and outperforming Federated Averaging without personalization. We evaluate these predictions on federated versions of the EMNIST, CIFAR-100, Shakespeare, and Stack Overflow datasets, where the experiments corroborate the theoretical predictions, suggesting such frameworks may provide a useful guide to practical algorithmic development.

preprint2022arXiv

PV-RCNN++: Semantical Point-Voxel Feature Interaction for 3D Object Detection

Large imbalance often exists between the foreground points (i.e., objects) and the background points in outdoor LiDAR point clouds. It hinders cutting-edge detectors from focusing on informative areas to produce accurate 3D object detection results. This paper proposes a novel object detection network by semantical point-voxel feature interaction, dubbed PV-RCNN++. Unlike most of existing methods, PV-RCNN++ explores the semantic information to enhance the quality of object detection. First, a semantic segmentation module is proposed to retain more discriminative foreground keypoints. Such a module will guide our PV-RCNN++ to integrate more object-related point-wise and voxel-wise features in the pivotal areas. Then, to make points and voxels interact efficiently, we utilize voxel query based on Manhattan distance to quickly sample voxel-wise features around keypoints. Such the voxel query will reduce the time complexity from O(N) to O(K), compared to the ball query. Further, to avoid being stuck in learning only local features, an attention-based residual PointNet module is designed to expand the receptive field to adaptively aggregate the neighboring voxel-wise features into keypoints. Extensive experiments on the KITTI dataset show that PV-RCNN++ achieves 81.60 $\%$ , 40.18 $\%$ , 68.21 $\%$ 3D mAP on Car, Pedestrian, and Cyclist, achieving comparable or even better performance to the state-of-the-arts.

preprint2022arXiv

When A Conventional Filter Meets Deep Learning: Basis Composition Learning on Image Filters

Image filters are fast, lightweight and effective, which make these conventional wisdoms preferable as basic tools in vision tasks. In practical scenarios, users have to tweak parameters multiple times to obtain satisfied results. This inconvenience heavily discounts the efficiency and user experience. We propose basis composition learning on single image filters to automatically determine their optimal formulas. The feasibility is based on a two-step strategy: first, we build a set of filtered basis (FB) consisting of approximations under selected parameter configurations; second, a dual-branch composition module is proposed to learn how the candidates in FB are combined to better approximate the target image. Our method is simple yet effective in practice; it renders filters to be user-friendly and benefits fundamental low-level vision problems including denoising, deraining and texture removal. Extensive experiments demonstrate that our method achieves an appropriate balance among the performance, time complexity and memory efficiency.

preprint2021arXiv

Accelerated, Optimal, and Parallel: Some Results on Model-Based Stochastic Optimization

We extend the Approximate-Proximal Point (aProx) family of model-based methods for solving stochastic convex optimization problems, including stochastic subgradient, proximal point, and bundle methods, to the minibatch and accelerated setting. To do so, we propose specific model-based algorithms and an acceleration scheme for which we provide non-asymptotic convergence guarantees, which are order-optimal in all problem-dependent constants and provide linear speedup in minibatch size, while maintaining the desirable robustness traits (e.g. to stepsize) of the aProx family. Additionally, we show improved convergence rates and matching lower bounds identifying new fundamental constants for "interpolation" problems, whose importance in statistical machine learning is growing; this, for example, gives a parallelization strategy for alternating projections. We corroborate our theoretical results with empirical testing to demonstrate the gains accurate modeling, acceleration, and minibatching provide.

preprint2020arXiv

Greedy Frank-Wolfe Algorithm for Exemplar Selection

In this paper, we consider the problem of selecting representatives from a data set for arbitrary supervised/unsupervised learning tasks. We identify a subset $S$ of a data set $A$ such that 1) the size of $S$ is much smaller than $A$ and 2) $S$ efficiently describes the entire data set, in a way formalized via convex optimization. In order to generate $|S| = k$ exemplars, our kernelizable algorithm, Frank-Wolfe Sparse Representation (FWSR), only needs to execute $\approx k$ iterations with a per-iteration cost that is quadratic in the size of $A$ . This is in contrast to other state of the art methods which need to execute until convergence with each iteration costing an extra factor of $d$ (dimension of the data). Moreover, we also provide a proof of linear convergence for our method. We support our results with empirical experiments; we test our algorithm against current methods in three different experimental setups on four different data sets. FWSR outperforms other exemplar finding methods both in speed and accuracy in almost all scenarios.

preprint2020arXiv

Handling Collocations in Hierarchical Latent Tree Analysis for Topic Modeling

Topic modeling has been one of the most active research areas in machine learning in recent years. Hierarchical latent tree analysis (HLTA) has been recently proposed for hierarchical topic modeling and has shown superior performance over state-of-the-art methods. However, the models used in HLTA have a tree structure and cannot represent the different meanings of multiword expressions sharing the same word appropriately. Therefore, we propose a method for extracting and selecting collocations as a preprocessing step for HLTA. The selected collocations are replaced with single tokens in the bag-of-words model before running HLTA. Our empirical evaluation shows that the proposed method led to better performance of HLTA on three of the four data sets tested.

preprint2016arXiv

Observation of Optical and Electrical In-plane Anisotropy in High-mobility Few-layer ZrTe5

Transition metal pentatelluride ZrTe5 is a versatile material in condensed-matter physics and has been intensively studied since the 1980s. The most fascinating feature of ZrTe5 is that it is a 3D Dirac semimetal which has linear energy dispersion in all three dimensions in momentum space. Structure-wise, ZrTe5 is a layered material held together by weak interlayer van der Waals force. The combination of its unique band structure and 2D atomic structure provides a fertile ground for more potential exotic physical phenomena in ZrTe5 related to 3D Dirac semimentals. However the physical properties of its few-layer form have yet to be thoroughly explored. Here we report strong optical and electrical in-plane anisotropy of mechanically exfoliated few-layer ZrTe5. Raman spectroscopy shows significant intensity change with sample orientations, and the behavior of angle-resolved phonon modes at the gamma point is explained by theoretical calculation. DC conductance measurement indicates a 50% of difference along different in-plane directions. The diminishing of resistivity anomaly in few-layer samples indicates the evolution of band structure with reduced thickness. Low-temperature Hall experiment sheds lights on more intrinsic anisotropic electrical transport, with hole mobility of 3,000 and 1,500 cm2/Vs along a-axis and c-axis respectively. Pronounced quantum oscillations in magneto-resistance are observed at low temperatures with highest electron mobility up to 44,000 cm2/Vs.

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Gary Cheng

Research overview

Is this your work?

8 publications

Approximate Function Evaluation via Multi-Armed Bandits

Federated Asymptotics: a model to compare federated learning algorithms

PV-RCNN++: Semantical Point-Voxel Feature Interaction for 3D Object Detection

When A Conventional Filter Meets Deep Learning: Basis Composition Learning on Image Filters

Accelerated, Optimal, and Parallel: Some Results on Model-Based Stochastic Optimization

Greedy Frank-Wolfe Algorithm for Exemplar Selection

Handling Collocations in Hierarchical Latent Tree Analysis for Topic Modeling

Observation of Optical and Electrical In-plane Anisotropy in High-mobility Few-layer ZrTe5

See the researcher in context

Building this map preview