Source author record

Paul Tupper

Paul Tupper appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Computation and Language math.NA Numerical Analysis

Catalog footprint

What is connected

3works

3topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2020arXiv

Generalizing Outside the Training Set: When Can Neural Networks Learn Identity Effects?

Often in language and other areas of cognition, whether two components of an object are identical or not determine whether it is well formed. We call such constraints identity effects. When developing a system to learn well-formedness from examples, it is easy enough to build in an identify effect. But can identity effects be learned from the data without explicit guidance? We provide a simple framework in which we can rigorously prove that algorithms satisfying simple criteria cannot make the correct inference. We then show that a broad class of algorithms including deep neural networks with standard architecture and training with backpropagation satisfy our criteria, dependent on the encoding of inputs. Finally, we demonstrate our theory with computational experiments in which we explore the effect of different input encodings on the ability of algorithms to generalize to novel inputs.

preprint2016arXiv

Which Learning Algorithms Can Generalize Identity-Based Rules to Novel Inputs?

We propose a novel framework for the analysis of learning algorithms that allows us to say when such algorithms can and cannot generalize certain patterns from training data to test data. In particular we focus on situations where the rule that must be learned concerns two components of a stimulus being identical. We call such a basis for discrimination an identity-based rule. Identity-based rules have proven to be difficult or impossible for certain types of learning algorithms to acquire from limited datasets. This is in contrast to human behaviour on similar tasks. Here we provide a framework for rigorously establishing which learning algorithms will fail at generalizing identity-based rules to novel stimuli. We use this framework to show that such algorithms are unable to generalize identity-based rules to novel inputs unless trained on virtually all possible inputs. We demonstrate these results computationally with a multilayer feedforward neural network.

preprint2014arXiv

Convergence of a Metropolized Integrator for Stochastic Differential Equations with Variable Diffusion Coefficient

We present an explicit method for simulating stochastic differential equations (SDEs) that have variable diffusion coefficients and satisfy the detailed balance condition with respect to a known equilibrium density. In Tupper and Yang (2012), we proposed a framework for such systems in which, instead of a diffusion coefficient and a drift coefficient, a modeller specifies a diffusion coefficient and a equilibrium density, and then assumes detailed balance with respect to this equilibrium density. We proposed a numerical method for such systems that works directly with the diffusion coefficient and equilibrium density, rather than the drift coefficient, and uses a Metropolis-Hastings rejection process to preserve the equilibrium density exactly. Here we show that the method is weakly convergent with order 1/2 for such systems with smooth coefficients. We perform numerical experiments demonstrating the convergence of the method for systems not covered by our theorem, including systems with discontinuous diffusion coefficients and equilibrium densities.