Source author record

Su Chen

Explore this author’s publications, collaborators and research topics. This catalog record has not been claimed by its author.

ResearcherUnclaimed source record

Databases math.NA

Catalog footprint

Research overview

2works

2topics

4close collaborators

Actions

Is this your work?

Open graph Browse papers

preprint2013arXiv

Higher order Sobol' indices

Sobol' indices measure the dependence of a high dimensional function on groups of variables defined on the unit cube $[0,1]^d$ . They are based on the ANOVA decomposition of functions, which is an $L^2$ decomposition. In this paper we discuss generalizations of Sobol' indices which yield $L^p$ measures of the dependence of $f$ on subsets of variables. Our interest is in values $p>2$ because then variable importance becomes more about reaching the extremes of $f$ . We introduce two methods. One based on higher order moments of the ANOVA terms and another based on higher order norms of a spectral decomposition of $f$ , including Fourier and Haar variants. Both of our generalizations have representations as integrals over $[0,1]^{kd}$ for $k\ge 1$ , allowing direct Monte Carlo or quasi-Monte Carlo estimation. We find that they are sensitive to different aspects of $f$ , and thus quantify different notions of variable importance.

preprint2012arXiv

Efficient Processing of k Nearest Neighbor Joins using MapReduce

k nearest neighbor join (kNN join), designed to find k nearest neighbors from a dataset S for every object in another dataset R, is a primitive operation widely adopted by many data mining applications. As a combination of the k nearest neighbor query and the join operation, kNN join is an expensive operation. Given the increasing volume of data, it is difficult to perform a kNN join on a centralized machine efficiently. In this paper, we investigate how to perform kNN join using MapReduce which is a well-accepted framework for data-intensive applications over clusters of computers. In brief, the mappers cluster objects into groups; the reducers perform the kNN join on each group of objects separately. We design an effective mapping mechanism that exploits pruning rules for distance filtering, and hence reduces both the shuffling and computational costs. To reduce the shuffling cost, we propose two approximate algorithms to minimize the number of replicas. Extensive experiments on our in-house cluster demonstrate that our proposed methods are efficient, robust and scalable.

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.