Source author record

Deep Ganguli

Deep Ganguli appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Computation and Language Machine Learning Neurons and Cognition Biological Physics

Catalog footprint

What is connected

4works

4topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2022arXiv

Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

We apply preference modeling and reinforcement learning from human feedback (RLHF) to finetune language models to act as helpful and harmless assistants. We find this alignment training improves performance on almost all NLP evaluations, and is fully compatible with training for specialized skills such as python coding and summarization. We explore an iterated online mode of training, where preference models and RL policies are updated on a weekly cadence with fresh human feedback data, efficiently improving our datasets and models. Finally, we investigate the robustness of RLHF training, and identify a roughly linear relation between the RL reward and the square root of the KL divergence between the policy and its initialization. Alongside our main results, we perform peripheral analyses on calibration, competing objectives, and the use of OOD detection, compare our models with human writers, and provide samples from our models using prompts appearing in recent related work.

preprint2021arXiv

Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models

On October 14th, 2020, researchers from OpenAI, the Stanford Institute for Human-Centered Artificial Intelligence, and other universities convened to discuss open research questions surrounding GPT-3, the largest publicly-disclosed dense language model at the time. The meeting took place under Chatham House Rules. Discussants came from a variety of research backgrounds including computer science, linguistics, philosophy, political science, communications, cyber policy, and more. Broadly, the discussion centered around two main questions: 1) What are the technical capabilities and limitations of large language models? 2) What are the societal effects of widespread use of large language models? Here, we provide a detailed summary of the discussion organized by the two themes above.

preprint2016arXiv

Neural and perceptual signatures of efficient sensory coding

The mammalian brain is a metabolically expensive device, and evolutionary pressures have presumably driven it to make productive use of its resources. For sensory areas, this concept has been expressed more formally as an optimality principle: the brain maximizes the information that is encoded about relevant sensory variables, given available resources. Here, we develop this efficiency principle for encoding a sensory variable with a heterogeneous population of noisy neurons, each responding to a particular range of values. The accuracy with which the population represents any particular value depends on the number of cells that respond to that value, their selectivity, and their response levels. We derive the optimal solution for these parameters in closed form, as a function of the probability of stimulus values encountered in the environment. This optimal neural population also imposes limitations on the ability of the organism to discriminate different values of the encoded variable. As a result, we predict an explicit relationship between the statistical properties of the environment, the allocation and selectivity of neurons within populations, and perceptual discriminability. We test this relationship for three visual and two auditory attributes, and find that it is remarkably consistent with existing data.

preprint2012arXiv

Implicit embedding of prior probabilities in optimally efficient neural populations

We examine how the prior probability distribution of a sensory variable in the environment influences the optimal allocation of neurons and spikes in a population that represents that variable. We start with a conventional response model, in which the spikes of each neuron are drawn from a Poisson distribution with a mean rate governed by an associated tuning curve. For this model, we approximate the Fisher information in terms of the density and amplitude of the tuning curves, under the assumption that tuning width varies inversely with cell density. We consider a family of objective functions based on the expected value, over the sensory prior, of a functional of the Fisher information. This family includes lower bounds on mutual information and perceptual discriminability as special cases. For all cases, we obtain a closed form expression for the optimum, in which the density and gain of the cells in the population are power law functions of the stimulus prior. Thus, the allocation of these resources is uniquely specified by the prior. Since perceptual discriminability may be expressed directly in terms of the Fisher information, it too will be a power law function of the prior. We show that these results hold for tuning curves of arbitrary shape and correlated neuronal variability. This framework thus provides direct and experimentally testable predictions regarding the relationship between sensory priors, tuning properties of neural representations, and perceptual discriminability.