Source author record

David Farhi

David Farhi appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

hep-ph Machine Learning cond-mat.dis-nn hep-ex hep-th Neural and Evolutionary Computing

Catalog footprint

What is connected

5works

6topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2022arXiv

Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer

Hyperparameter (HP) tuning in deep learning is an expensive process, prohibitively so for neural networks (NNs) with billions of parameters. We show that, in the recently discovered Maximal Update Parametrization (muP), many optimal HPs remain stable even as model size changes. This leads to a new HP tuning paradigm we call muTransfer: parametrize the target model in muP, tune the HP indirectly on a smaller model, and zero-shot transfer them to the full-sized model, i.e., without directly tuning the latter at all. We verify muTransfer on Transformer and ResNet. For example, 1) by transferring pretraining HPs from a model of 13M parameters, we outperform published numbers of BERT-large (350M parameters), with a total tuning cost equivalent to pretraining BERT-large once; 2) by transferring from 40M parameters, we outperform published numbers of the 6.7B GPT-3 model, with tuning cost only 7% of total pretraining cost. A Pytorch implementation of our technique can be found at github.com/microsoft/mup and installable via `pip install mup`.

preprint2019arXiv

Dota 2 with Large Scale Deep Reinforcement Learning

On April 13th, 2019, OpenAI Five became the first AI system to defeat the world champions at an esports game. The game of Dota 2 presents novel challenges for AI systems such as long time horizons, imperfect information, and complex, continuous state-action spaces, all challenges which will become increasingly central to more capable AI systems. OpenAI Five leveraged existing reinforcement learning techniques, scaled to learn from batches of approximately 2 million frames every 2 seconds. We developed a distributed training system and tools for continual training which allowed us to train OpenAI Five for 10 months. By defeating the Dota 2 world champion (Team OG), OpenAI Five demonstrates that self-play reinforcement learning can achieve superhuman performance on a difficult task.

preprint2016arXiv

A direct approach to quantum tunneling

The decay rates of quasistable states in quantum field theories are usually calculated using instanton methods. Standard derivations of these methods rely in a crucial way upon deformations and analytic continuations of the physical potential, and on the saddle point approximation. While the resulting procedure can be checked against other semi-classical approaches in some one-dimensional cases, it is challenging to trace the role of the relevant physical scales, and any intuitive handle on the precision of the approximations involved are at best obscure. In this paper, we use a physical definition of the tunneling probability to derive a formula for the decay rate in both quantum mechanics and quantum field theory directly from the Minkowski path integral, without reference to unphysical deformations of the potential. There are numerous benefits to this approach, from non-perturbative applications to precision calculations and aesthetic simplicity.

preprint2015arXiv

Streamlining resummed QCD calculations using Monte Carlo integration

Some of the most arduous and error-prone aspects of precision resummed calculations are related to the partonic hard process, having nothing to do with the resummation. In particular, interfacing to parton-distribution functions, combining various channels, and performing the phase space integration can be limiting factors in completing calculations. Conveniently, however, most of these tasks are already automated in many Monte Carlo programs, such as MadGraph, Alpgen or Sherpa. In this paper, we show how such programs can be used to produce distributions of partonic kinematics with associated color structures representing the hard factor in a resummed distribution. These distributions can then be used to weight convolutions of jet, soft and beam functions producing a complete resummed calculation. In fact, only around 1000 unweighted events are necessary to produce precise distributions. A number of examples and checks are provided, including $e^+e^-$ two- and four-jet event shapes, $n$-jettiness and jet-mass related observables at hadron colliders. Attached code can be used to modify MadGraph to export the relevant leading-order hard functions and color structures for arbitrary processes.

preprint2014arXiv

Quantifying the power of multiple event interpretations

A number of methods have been proposed recently which exploit multiple highly-correlated interpretations of events, or of jets within an event. For example, Qjets reclusters a jet multiple times and telescoping jets uses multiple cone sizes. Previous work has employed these methods in pseudo-experimental analyses and found that, with a simplified statistical treatment, they give sizable improvements over traditional methods. In this paper, the improvement gain from multiple event interpretations is explored with methods much closer to those used in real experiments. To this end, we derive a generalized extended maximum likelihood procedure. We study the significance improvement in Higgs to bb with both this method and the simplified method from previous analysis. With either method, we find that using multiple jet radii can provide substantial benefit over a single radius. Another concern we address is that multiple event interpretations might be exploiting similar information to that already present in the standard kinematic variables. By examining correlations between kinematic variables commonly used in LHC analyses and invariant masses obtained with multiple jet reconstructions, we find that using multiple radii is still helpful even on top of standard kinematic variables when combined with boosted decision trees. These results suggest that including multiple event interpretations in a realistic search for Higgs to bb would give additional sensitivity over traditional approaches.