BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Sanjay Chawla

Sanjay Chawla appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Machine LearningDatabasesInformation RetrievalSocial and Information NetworksComputer VisionData Structures and AlgorithmsArtificial IntelligenceComputation and LanguageNetworking and Internet ArchitectureNeurons and Cognitionphysics.soc-phq-fin.EC

Catalog footprint

What is connected

21works
12topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

21 published item(s)

preprint2020arXiv

RHEEMix in the Data Jungle: A Cost-based Optimizer for Cross-platform Systems

In pursuit of efficient and scalable data analytics, the insight that "one size does not fit all" has given rise to a plethora of specialized data processing platforms and today's complex data analytics are moving beyond the limits of a single platform. In this paper, we present the cost-based optimizer of Rheem, an open-source cross-platform system that copes with these new requirements. The optimizer allocates the subtasks of data analytic tasks to the most suitable platforms. Our main contributions are: (i)~a mechanism based on graph transformations to explore alternative execution strategies; (ii)~a novel graph-based approach to efficiently plan data movement among subtasks and platforms; and (iii)~an efficient plan enumeration algorithm, based on a novel enumeration algebra. We extensively evaluate our optimizer under diverse real tasks. The results show that our optimizer is capable of selecting the most efficient platform combination for a given task, freeing data analysts from the need to choose and orchestrate platforms. In addition, our optimizer allows tasks to run more than one order of magnitude faster by using multiple platforms instead of a single platform.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Sat2Graph: Road Graph Extraction through Graph-Tensor Encoding

Inferring road graphs from satellite imagery is a challenging computer vision task. Prior solutions fall into two categories: (1) pixel-wise segmentation-based approaches, which predict whether each pixel is on a road, and (2) graph-based approaches, which predict the road graph iteratively. We find that these two approaches have complementary strengths while suffering from their own inherent limitations. In this paper, we propose a new method, Sat2Graph, which combines the advantages of the two prior categories into a unified framework. The key idea in Sat2Graph is a novel encoding scheme, graph-tensor encoding (GTE), which encodes the road graph into a tensor representation. GTE makes it possible to train a simple, non-recurrent, supervised model to predict a rich set of features that capture the graph structure directly from an image. We evaluate Sat2Graph using two large datasets. We find that Sat2Graph surpasses prior methods on two widely used metrics, TOPO and APLS. Furthermore, whereas prior work only infers planar road graphs, our approach is capable of inferring stacked roads (e.g., overpasses), and does so robustly.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Unravelling the Architecture of Membrane Proteins with Conditional Random Fields

In this paper, we will show that the recently introduced graphical model: Conditional Random Fields (CRF) provides a template to integrate micro-level information about biological entities into a mathematical model to understand their macro-level behavior. More specifically, we will apply the CRF model to an important classification problem in protein science, namely the secondary structure prediction of proteins based on the observed primary structure. A comparison on benchmark data sets against twenty-eight other methods shows that not only does the CRF model lead to extremely accurate predictions but the modular nature of the model and the freedom to integrate disparate, overlapping and non-independent sources of information, makes the model an extremely versatile tool to potentially solve many other problems in bioinformatics.

0 citations0 reviewsNo code linkedOpen access
preprint2019arXiv

RoadTagger: Robust Road Attribute Inference with Graph Neural Networks

Inferring road attributes such as lane count and road type from satellite imagery is challenging. Often, due to the occlusion in satellite imagery and the spatial correlation of road attributes, a road attribute at one position on a road may only be apparent when considering far-away segments of the road. Thus, to robustly infer road attributes, the model must integrate scattered information and capture the spatial correlation of features along roads. Existing solutions that rely on image classifiers fail to capture this correlation, resulting in poor accuracy. We find this failure is caused by a fundamental limitation -- the limited effective receptive field of image classifiers. To overcome this limitation, we propose RoadTagger, an end-to-end architecture which combines both Convolutional Neural Networks (CNNs) and Graph Neural Networks (GNNs) to infer road attributes. The usage of graph neural networks allows information propagation on the road network graph and eliminates the receptive field limitation of image classifiers. We evaluate RoadTagger on both a large real-world dataset covering 688 km^2 area in 20 U.S. cities and a synthesized micro-dataset. In the evaluation, RoadTagger improves inference accuracy over the CNN image classifier based approaches. RoadTagger also demonstrates strong robustness against different disruptions in the satellite imagery and the ability to learn complicated inductive rules for aggregating scattered information along the road network.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

A Robust Framework for Classifying Evolving Document Streams in an Expert-Machine-Crowd Setting

An emerging challenge in the online classification of social media data streams is to keep the categories used for classification up-to-date. In this paper, we propose an innovative framework based on an Expert-Machine-Crowd (EMC) triad to help categorize items by continuously identifying novel concepts in heterogeneous data streams often riddled with outliers. We unify constrained clustering and outlier detection by formulating a novel optimization problem: COD-Means. We design an algorithm to solve the COD-Means problem and show that COD-Means will not only help detect novel categories but also seamlessly discover human annotation errors and improve the overall quality of the categorization process. Experiments on diverse real data sets demonstrate that our approach is both effective and efficient.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Eigenvector dynamics under perturbation of modular networks

Rotation dynamics of eigenvectors of modular network adjacency matrices under random perturbations are presented. In the presence of $q$ communities, the number of eigenvectors corresponding to the $q$ largest eigenvalues form a "community" eigenspace and rotate together, but separately from that of the "bulk" eigenspace spanned by all the other eigenvectors. Using this property, the number of modules or clusters in a network can be estimated in an algorithm-independent way. A general argument and derivation for the theoretical detectability limit for sparse modular networks with $q$ communities is presented, beyond which modularity persists in the system but cannot be detected. It is shown that for detecting the clusters or modules using the adjacency matrix, there is a "band" in which it is hard to detect the clusters even before the theoretical detectability limit is reached, and for which the theoretically predicted detectability limit forms the sufficient upper bound. Analytic estimations of these bounds are presented, and empirically demonstrated.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Inferring the contiguity matrix for spatial autoregressive analysis with applications to house price prediction

Inference methods in traditional statistics, machine learning and data mining assume that data is generated from an independent and identically distributed (iid) process. Spatial data exhibits behavior for which the iid assumption must be relaxed. For example, the standard approach in spatial regression is to assume the existence of a contiguity matrix which captures the spatial autoregressive properties of the data. However all spatial methods, till now, have assumed that the contiguity matrix is given apriori or can be estimated by using a spatial similarity function. In this paper we propose a convex optimization formulation to solve the spatial autoregressive regression (SAR) model in which both the contiguity matrix and the non-spatial regression parameters are unknown and inferred from the data. We solve the problem using the alternating direction method of multipliers (ADMM) which provides a solution which is both robust and efficient. While our approach is general we use data from housing markets of Boston and Sydney to both guide the analysis and validate our results. A novel side effect of our approach is the automatic discovery of spatial clusters which translate to submarkets in the housing data sets.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Scalable Constrained Clustering: A Generalized Spectral Method

We present a principled spectral approach to the well-studied constrained clustering problem. It reduces clustering to a generalized eigenvalue problem on Laplacians. The method works in nearly-linear time and provides concrete guarantees for the quality of the clusters, at least for the case of 2-way partitioning. In practice this translates to a very fast implementation that consistently outperforms existing spectral approaches. We support this claim with experiments on various data sets: our approach recovers correct clusters in examples where previous methods fail, and handles data sets with millions of data points - two orders of magnitude larger than before.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Scalable Constrained Clustering: A Generalized Spectral Method

We present a simple spectral approach to the well-studied constrained clustering problem. It captures constrained clustering as a generalized eigenvalue problem with graph Laplacians. The algorithm works in nearly-linear time and provides concrete guarantees for the quality of the clusters, at least for the case of 2-way partitioning. In practice this translates to a very fast implementation that consistently outperforms existing spectral approaches both in speed and quality.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Discovering the Network Backbone from Traffic Activity Data

We introduce a new computational problem, the BackboneDiscovery problem, which encapsulates both functional and structural aspects of network analysis. While the topology of a typical road network has been available for a long time (e.g., through maps), it is only recently that fine-granularity functional (activity and usage) information about the network (like source-destination traffic information) is being collected and is readily available. The combination of functional and structural information provides an efficient way to explore and understand usage patterns of networks and aid in design and decision making. We propose efficient algorithms for the BackboneDiscovery problem including a novel use of edge centrality. We observe that for many real world networks, our algorithm produces a backbone with a small subset of the edges that support a large percentage of the network activity.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

On inferring structural connectivity from brain functional-MRI data

The anatomical structure of the brain can be observed via non-invasive techniques such as diffusion imaging. However, these are imperfect because they miss connections that are actually known to exist, especially long range inter-hemispheric ones. In this paper we formulate the inverse problem of inferring the structural connectivity of brain networks from experimentally observed functional connectivity via functional Magnetic Resonance Imaging (fMRI), by formulating it as a convex optimization problem. We show that structural connectivity can be modeled as an optimal sparse representation derived from the much denser functional connectivity in the human brain. Using only the functional connectivity data as input, we present (a) an optimization problem that models constraints based on known physiological observations, and (b) an ADMM algorithm for solving it. The algorithm not only recovers the known structural connectivity of the brain, but is also able to robustly predict the long range inter-hemispheric connections missed by DSI or DTI, including a very good match with experimentally observed quantitative distributions of the weights/strength of anatomical connections. We demonstrate results on both synthetic model data and a fine-scale 998 node cortical dataset, and discuss applications to other complex network domains where retrieving effective structure from functional signatures are important.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Parameter Database : Data-centric Synchronization for Scalable Machine Learning

We propose a new data-centric synchronization framework for carrying out of machine learning (ML) tasks in a distributed environment. Our framework exploits the iterative nature of ML algorithms and relaxes the application agnostic bulk synchronization parallel (BSP) paradigm that has previously been used for distributed machine learning. Data-centric synchronization complements function-centric synchronization based on using stale updates to increase the throughput of distributed ML computations. Experiments to validate our framework suggest that we can attain substantial improvement over BSP while guaranteeing sequential correctness of ML tasks.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Integer Programming Relaxations for Integrated Clustering and Outlier Detection

In this paper we present methods for exemplar based clustering with outlier selection based on the facility location formulation. Given a distance function and the number of outliers to be found, the methods automatically determine the number of clusters and outliers. We formulate the problem as an integer program to which we present relaxations that allow for solutions that scale to large data sets. The advantages of combining clustering and outlier selection include: (i) the resulting clusters tend to be compact and semantically coherent (ii) the clusters are more robust against data perturbations and (iii) the outliers are contextualised by the clusters and more interpretable, i.e. it is easier to distinguish between outliers which are the result of data errors from those that may be indicative of a new pattern emergent in the data. We present and contrast three relaxations to the integer program formulation: (i) a linear programming formulation (LP) (ii) an extension of affinity propagation to outlier detection (APOC) and (iii) a Lagrangian duality based formulation (LD). Evaluation on synthetic as well as real data shows the quality and scalability of these different methods.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Network Traffic Decomposition for Anomaly Detection

In this paper we focus on the detection of network anomalies like Denial of Service (DoS) attacks and port scans in a unified manner. While there has been an extensive amount of research in network anomaly detection, current state of the art methods are only able to detect one class of anomalies at the cost of others. The key tool we will use is based on the spectral decomposition of a trajectory/hankel matrix which is able to detect deviations from both between and within correlation present in the observed network traffic data. Detailed experiments on synthetic and real network traces shows a significant improvement in detection capability over competing approaches. In the process we also address the issue of robustness of anomaly detection systems in a principled fashion.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Sleep Analytics and Online Selective Anomaly Detection

We introduce a new problem, the Online Selective Anomaly Detection (OSAD), to model a specific scenario emerging from research in sleep science. Scientists have segmented sleep into several stages and stage two is characterized by two patterns (or anomalies) in the EEG time series recorded on sleep subjects. These two patterns are sleep spindle (SS) and K-complex. The OSAD problem was introduced to design a residual system, where all anomalies (known and unknown) are detected but the system only triggers an alarm when non-SS anomalies appear. The solution of the OSAD problem required us to combine techniques from both machine learning and control theory. Experiments on data from real subjects attest to the effectiveness of our approach.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Feature Graph Architectures

In this article we propose feature graph architectures (FGA), which are deep learning systems employing a structured initialisation and training method based on a feature graph which facilitates improved generalisation performance compared with a standard shallow architecture. The goal is to explore alternative perspectives on the problem of deep network training. We evaluate FGA performance for deep SVMs on some experimental datasets, and show how generalisation and stability results may be derived for these models. We describe the effect of permutations on the model accuracy, and give a criterion for the optimal permutation in terms of feature correlations. The experimental results show that the algorithm produces robust and significant test set improvements over a standard shallow SVM training method for a range of datasets. These gains are achieved with a moderate increase in time complexity.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

Large Scale Spectral Clustering Using Approximate Commute Time Embedding

Spectral clustering is a novel clustering method which can detect complex shapes of data clusters. However, it requires the eigen decomposition of the graph Laplacian matrix, which is proportion to $O(n^3)$ and thus is not suitable for large scale systems. Recently, many methods have been proposed to accelerate the computational time of spectral clustering. These approximate methods usually involve sampling techniques by which a lot information of the original data may be lost. In this work, we propose a fast and accurate spectral clustering approach using an approximate commute time embedding, which is similar to the spectral embedding. The method does not require using any sampling technique and computing any eigenvector at all. Instead it uses random projection and a linear time solver to find the approximate embedding. The experiments in several synthetic and real datasets show that the proposed approach has better clustering quality and is faster than the state-of-the-art approximate spectral clustering methods.

0 citations0 reviewsNo code linkedOpen access
preprint2012arXiv

SparseDTW: A Novel Approach to Speed up Dynamic Time Warping

We present a new space-efficient approach, (SparseDTW), to compute the Dynamic Time Warping (DTW) distance between two time series that always yields the optimal result. This is in contrast to other known approaches which typically sacrifice optimality to attain space efficiency. The main idea behind our approach is to dynamically exploit the existence of similarity and/or correlation between the time series. The more the similarity between the time series the less space required to compute the DTW between them. To the best of our knowledge, all other techniques to speedup DTW, impose apriori constraints and do not exploit similarity characteristics that may be present in the data. We conduct experiments and demonstrate that SparseDTW outperforms previous approaches.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

An index for regular expression queries: Design and implementation

The like regular expression predicate has been part of the SQL standard since at least 1989. However, despite its popularity and wide usage, database vendors provide only limited indexing support for regular expression queries which almost always require a full table scan. In this paper we propose a rigorous and robust approach for providing indexing support for regular expression queries. Our approach consists of formulating the indexing problem as a combinatorial optimization problem. We begin with a database, abstracted as a collection of strings. From this data set we generate a query workload. The input to the optimization problem is the database and the workload. The output is a set of multigrams (substrings) which can be used as keys to records which satisfy the query workload. The multigrams can then be integrated with the data structure (like B+ trees) to provide indexing support for the queries. We provide a deterministic and a randomized approximation algorithm (with provable guarantees) to solve the optimization problem. Extensive experiments on synthetic data sets demonstrate that our approach is accurate and efficient. We also present a case study on PROSITE patterns - which are complex regular expression signatures for classes of proteins. Again, we are able to demonstrate the utility of our indexing approach in terms of accuracy and efficiency. Thus, perhaps for the first time, there is a robust and practical indexing mechanism for an important class of database queries.

0 citations0 reviewsNo code linkedOpen access
preprint2011arXiv

Online Anomaly Detection Systems Using Incremental Commute Time

Commute Time Distance (CTD) is a random walk based metric on graphs. CTD has found widespread applications in many domains including personalized search, collaborative filtering and making search engines robust against manipulation. Our interest is inspired by the use of CTD as a metric for anomaly detection. It has been shown that CTD can be used to simultaneously identify both global and local anomalies. Here we propose an accurate and efficient approximation for computing the CTD in an incremental fashion in order to facilitate real-time applications. An online anomaly detection algorithm is designed where the CTD of each new arriving data point to any point in the current graph can be estimated in constant time ensuring a real-time response. Moreover, the proposed approach can also be applied in many other applications that utilize commute time distance.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Angle Tree: Nearest Neighbor Search in High Dimensions with Low Intrinsic Dimensionality

We propose an extension of tree-based space-partitioning indexing structures for data with low intrinsic dimensionality embedded in a high dimensional space. We call this extension an Angle Tree. Our extension can be applied to both classical kd-trees as well as the more recent rp-trees. The key idea of our approach is to store the angle (the "dihedral angle") between the data region (which is a low dimensional manifold) and the random hyperplane that splits the region (the "splitter"). We show that the dihedral angle can be used to obtain a tight lower bound on the distance between the query point and any point on the opposite side of the splitter. This in turn can be used to efficiently prune the search space. We introduce a novel randomized strategy to efficiently calculate the dihedral angle with a high degree of accuracy. Experiments and analysis on real and synthetic data sets shows that the Angle Tree is the most efficient known indexing structure for nearest neighbor queries in terms of preprocessing and space usage while achieving high accuracy and fast search time.

0 citations0 reviewsNo code linkedOpen access