Source author record

Marc W. Howard

Marc W. Howard appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record

Machine Learning Neurons and Cognition Artificial Intelligence Computation and Language Neural and Evolutionary Computing

Catalog footprint

What is connected

5works

5topics

4close collaborators

Actions

Connect this record

Open graph Browse works

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

preprint2026arXiv

Hierarchical temporal receptive windows and zero-shot timescale generalization in biologically constrained scale-invariant deep networks

Human cognition integrates information across nested timescales. While the cortex exhibits hierarchical Temporal Receptive Windows (TRWs), local circuits often display heterogeneous time constants. To reconcile this, we trained biologically constrained deep networks, based on scale-invariant hippocampal time cells, on a language classification task mimicking the hierarchical structure of language (e.g., 'letters' forming 'words'). First, using a feedforward model (SITHCon), we found that a hierarchy of TRWs emerged naturally across layers, despite the network having an identical spectrum of time constants within layers. We then distilled these inductive priors into a biologically plausible recurrent architecture, SITH-RNN. Training a sequence of architectures ranging from generic RNNs to this restricted subset showed that the scale-invariant SITH-RNN learned faster with orders-of-magnitude fewer parameters, and generalized zero-shot to out-of-distribution timescales. These results suggest the brain employs scale-invariant, sequential priors - coding "what" happened "when" - making recurrent networks with such priors particularly well-suited to describe human cognition.

preprint2022arXiv

A deep convolutional neural network that is invariant to time rescaling

Human learners can readily understand speech, or a melody, when it is presented slower or faster than usual. Although deep convolutional neural networks (CNNs) are extremely powerful in extracting information from time series, they require explicit training to generalize to different time scales. This paper presents a deep CNN that incorporates a temporal representation inspired by recent findings from neuroscience. In the mammalian brain, time is represented by populations of neurons with temporal receptive fields. Critically, the peaks of the receptive fields form a geometric series, such that the population codes a set of temporal basis functions over log time. Because memory for the recent past is a function of log time, rescaling the input results in translation of the memory. The Scale-Invariant Temporal History Convolution network (SITHCon) builds a convolutional layer over this logarithmically-distributed temporal memory. A max-pool operation results in a network that is invariant to rescalings of time modulo edge effects. We compare performance of SITHCon to a Temporal Convolution Network (TCN). Although both networks can learn classification and regression problems on both univariate and multivariate time series f(t), only SITHCon generalizes to rescalings f(at). This property, inspired by findings from contemporary neuroscience and consistent with findings from cognitive psychology, may enable networks that learn with fewer training examples, fewer weights and that generalize more robustly to out of sample data.

preprint2022arXiv

Formal models of memory based on temporally-varying representations

The idea that memory behavior relies on a gradually-changing internal state has a long history in mathematical psychology. This chapter traces this line of thought from statistical learning theory in the 1950s, through distributed memory models in the latter part of the 20th century and early part of the 21st century through to modern models based on a scale-invariant temporal history. We discuss the neural phenomena consistent with this form of representation and sketch the kinds of cognitive models that can be constructed using it and connections with formal models of various memory tasks.

preprint2020arXiv

Cognitive computation using neural representations of time and space in the Laplace domain

Memory for the past makes use of a record of what happened when---a function over past time. Time cells in the hippocampus and temporal context cells in the entorhinal cortex both code for events as a function of past time, but with very different receptive fields. Time cells in the hippocampus can be understood as a compressed estimate of events as a function of the past. Temporal context cells in the entorhinal cortex can be understood as the Laplace transform of that function, respectively. Other functional cell types in the hippocampus and related regions, including border cells, place cells, trajectory coding, splitter cells, can be understood as coding for functions over space or past movements or their Laplace transforms. More abstract quantities, like distance in an abstract conceptual space or numerosity could also be mapped onto populations of neurons coding for the Laplace transform of functions over those variables. Quantitative cognitive models of memory and evidence accumulation can also be specified in this framework allowing constraints from both behavior and neurophysiology. More generally, the computational power of the Laplace domain could be important for efficiently implementing data-independent operators, which could serve as a basis for neural models of a very broad range of cognitive computations.

preprint2013arXiv

Optimally fuzzy temporal memory

Any learner with the ability to predict the future of a structured time-varying signal must maintain a memory of the recent past. If the signal has a characteristic timescale relevant to future prediction, the memory can be a simple shift register---a moving window extending into the past, requiring storage resources that linearly grows with the timescale to be represented. However, an independent general purpose learner cannot a priori know the characteristic prediction-relevant timescale of the signal. Moreover, many naturally occurring signals show scale-free long range correlations implying that the natural prediction-relevant timescale is essentially unbounded. Hence the learner should maintain information from the longest possible timescale allowed by resource availability. Here we construct a fuzzy memory system that optimally sacrifices the temporal accuracy of information in a scale-free fashion in order to represent prediction-relevant information from exponentially long timescales. Using several illustrative examples, we demonstrate the advantage of the fuzzy memory system over a shift register in time series forecasting of natural signals. When the available storage resources are limited, we suggest that a general purpose learner would be better off committing to such a fuzzy memory system.