Source author record

Aurko Roy

Aurko Roy appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Machine Learning Computation and Language Computational Complexity Discrete Mathematics

Catalog footprint

What is connected

4works

4topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2022arXiv

N-Grammer: Augmenting Transformers with latent n-grams

Transformer models have recently emerged as one of the foundational models in natural language processing, and as a byproduct, there is significant recent interest and investment in scaling these models. However, the training and inference costs of these large Transformer language models are prohibitive, thus necessitating more research in identifying more efficient variants. In this work, we propose a simple yet effective modification to the Transformer architecture inspired by the literature in statistical language modeling, by augmenting the model with n-grams that are constructed from a discrete latent representation of the text sequence. We evaluate our model, the N-Grammer on language modeling on the C4 data-set as well as text classification on the SuperGLUE data-set, and find that it outperforms several strong baselines such as the Transformer and the Primer. We open-source our model for reproducibility purposes in Jax.

preprint2016arXiv

Hierarchical Clustering via Spreading Metrics

We study the cost function for hierarchical clusterings introduced by [arXiv:1510.05043] where hierarchies are treated as first-class objects rather than deriving their cost from projections into flat clusters. It was also shown in [arXiv:1510.05043] that a top-down algorithm returns a hierarchical clustering of cost at most $O\left(α_n \log n\right)$ times the cost of the optimal hierarchical clustering, where $α_n$ is the approximation ratio of the Sparsest Cut subroutine used. Thus using the best known approximation algorithm for Sparsest Cut due to Arora-Rao-Vazirani, the top down algorithm returns a hierarchical clustering of cost at most $O\left(\log^{3/2} n\right)$ times the cost of the optimal solution. We improve this by giving an $O(\log{n})$-approximation algorithm for this problem. Our main technical ingredients are a combinatorial characterization of ultrametrics induced by this cost function, deriving an Integer Linear Programming (ILP) formulation for this family of ultrametrics, and showing how to iteratively round an LP relaxation of this formulation by using the idea of \emph{sphere growing} which has been extensively used in the context of graph partitioning. We also prove that our algorithm returns an $O(\log{n})$-approximate hierarchical clustering for a generalization of this cost function also studied in [arXiv:1510.05043]. Experiments show that the hierarchies found by using the ILP formulation as well as our rounding algorithm often have better projections into flat clusters than the standard linkage based algorithms. We also give constant factor inapproximability results for this problem.

preprint2016arXiv

The matching problem has no small symmetric SDP

Yannakakis showed that the matching problem does not have a small symmetric linear program. Rothvoß recently proved that any, not necessarily symmetric, linear program also has exponential size. It is natural to ask whether the matching problem can be expressed compactly in a framework such as semidefinite programming (SDP) that is more powerful than linear programming but still allows efficient optimization. We answer this question negatively for symmetric SDPs: any symmetric SDP for the matching problem has exponential size. We also show that an O(k)-round Lasserre SDP relaxation for the metric traveling salesperson problem yields at least as good an approximation as any symmetric SDP relaxation of size $n^k$. The key technical ingredient underlying both these results is an upper bound on the degree needed to derive polynomial identities that hold over the space of matchings or traveling salesperson tours.

preprint2015arXiv

Deterministic polynomial factoring under the assumption of the Extended Riemann Hypothesis (ERH)

We consider the problem of deterministically factoring a univariate polynomial over a finite field under the assumption of the Extended Riemann Hypothesis (ERH). This work builds upon the line of approach first explored by Gao in $2001$. The general approach has been to implicitly construct a graph with the roots as vertices and the edges formed by some polynomial time computable relation. The algorithm then fails to factor a polynomial if this associated graph turns out to be \emph{regular}. In the first part of our work we strengthen the edge relation so that the resulting set of graphs we obtain are subgraphs of Gao's graphs, all of which must be \emph{regular}. In the second part of our work we strengthen the regularity condition of these graphs. This is accomplished by finding a parallel between their algorithms and the $1$-dimensional Weisfeiler-Leman algorithm for solving the Graph Isomorphism problem. We observe that the general principle behind their algorithms is to separate the roots by computing the $1$-dimensional Weisfeiler-Leman approximation to the orbits of this graph. This leads us to the natural question of whether this approximation may be improved. We then go on to show how to implicitly compute the $2$-dimensional Weisfeiler-Leman approximation of the orbits of these graphs. The polynomials that our algorithm fails to factor form graphs that are \emph{strongly regular} and their set of adjacency matrices forms a combinatorial structure called an \emph{Association scheme}.