Source author record

Jiecao Chen

Jiecao Chen appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Data Structures and Algorithms Computation and Language Information Retrieval Machine Learning

Catalog footprint

What is connected

5works

4topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2022arXiv

Multi-Task Retrieval-Augmented Text Generation with Relevance Sampling

This paper studies multi-task training of retrieval-augmented generation models for knowledge-intensive tasks. We propose to clean the training set by utilizing a distinct property of knowledge-intensive generation: The connection of query-answer pairs to items in the knowledge base. We filter training examples via a threshold of confidence on the relevance labels, whether a pair is answerable by the knowledge base or not. We train a single Fusion-in-Decoder (FiD) generator on seven combined tasks of the KILT benchmark. The experimental results suggest that our simple yet effective approach substantially improves competitive baselines on two strongly imbalanced tasks; and shows either smaller improvements or no significant regression on the remaining tasks. Furthermore, we demonstrate our multi-task training with relevance label sampling scales well with increased model capacity and achieves state-of-the-art results in five out of seven KILT tasks.

preprint2019arXiv

Sampled Softmax with Random Fourier Features

The computational cost of training with softmax cross entropy loss grows linearly with the number of classes. For the settings where a large number of classes are involved, a common method to speed up training is to sample a subset of classes and utilize an estimate of the loss gradient based on these classes, known as the sampled softmax method. However, the sampled softmax provides a biased estimate of the gradient unless the samples are drawn from the exact softmax distribution, which is again expensive to compute. Therefore, a widely employed practical approach involves sampling from a simpler distribution in the hope of approximating the exact softmax distribution. In this paper, we develop the first theoretical understanding of the role that different sampling distributions play in determining the quality of sampled softmax. Motivated by our analysis and the work on kernel-based sampling, we propose the Random Fourier Softmax (RF-softmax) method that utilizes the powerful Random Fourier Features to enable more efficient and accurate sampling from an approximate softmax distribution. We show that RF-softmax leads to low bias in estimation in terms of both the full softmax distribution and the full softmax gradient. Furthermore, the cost of RF-softmax scales only logarithmically with the number of classes.

preprint2016arXiv

Improved Algorithms for Distributed Entropy Monitoring

Modern data management systems often need to deal with massive, dynamic and inherently distributed data sources. We collect the data using a distributed network, and at the same time try to maintain a global view of the data at a central coordinator using a minimal amount of communication. Such applications have been captured by the distributed monitoring model which has attracted a lot of attention in recent years. In this paper we investigate the monitoring of the entropy functions, which are very useful in network monitoring applications such as detecting distributed denial-of-service attacks. Our results improve the previous best results by Arackaparambil et al. [2]. Our technical contribution also includes implementing the celebrated AMS sampling method (by Alon et al. [1]) in the distributed monitoring model, which could be of independent interest.

preprint2016arXiv

Submodular Maximization over Sliding Windows

In this paper we study the extraction of representative elements in the data stream model in the form of submodular maximization. Different from the previous work on streaming submodular maximization, we are interested only in the recent data, and study the maximization problem over sliding windows. We provide a general reduction from the sliding window model to the standard streaming model, and thus our approach works for general constraints as long as there is a corresponding streaming algorithm in the standard streaming model. As a consequence, we obtain the first algorithms in the sliding window model for maximizing a monotone/non-monotone submodular function under cardinality and matroid constraints. We also propose several heuristics and show their efficiency in real-world datasets.

preprint2014arXiv

A Sketching Algorithm for Spectral Graph Sparsification

We study the problem of compressing a weighted graph $G$ on $n$ vertices, building a "sketch" $H$ of $G$, so that given any vector $x \in \mathbb{R}^n$, the value $x^T L_G x$ can be approximated up to a multiplicative $1+ε$ factor from only $H$ and $x$, where $L_G$ denotes the Laplacian of $G$. One solution to this problem is to build a spectral sparsifier $H$ of $G$, which, using the result of Batson, Spielman, and Srivastava, consists of $O(n ε^{-2})$ reweighted edges of $G$ and has the property that simultaneously for all $x \in \mathbb{R}^n$, $x^T L_H x = (1 \pm ε) x^T L_G x$. The $O(n ε^{-2})$ bound is optimal for spectral sparsifiers. We show that if one is interested in only preserving the value of $x^T L_G x$ for a {\it fixed} $x \in \mathbb{R}^n$ (specified at query time) with high probability, then there is a sketch $H$ using only $\tilde{O}(n ε^{-1.6})$ bits of space. This is the first data structure achieving a sub-quadratic dependence on $ε$. Our work builds upon recent work of Andoni, Krauthgamer, and Woodruff who showed that $\tilde{O}(n ε^{-1})$ bits of space is possible for preserving a fixed {\it cut query} (i.e., $x\in \{0,1\}^n$) with high probability; here we show that even for a general query vector $x \in \mathbb{R}^n$, a sub-quadratic dependence on $ε$ is possible. Our result for Laplacians is in sharp contrast to sketches for general $n \times n$ positive semidefinite matrices $A$ with $O(\log n)$ bit entries, for which even to preserve the value of $x^T A x$ for a fixed $x \in \mathbb{R}^n$ (specified at query time) up to a $1+ε$ factor with constant probability, we show an $Ω(n ε^{-2})$ lower bound.