BZPEERReading, trust and collaboration for research
Menu

Discover

GraphFollow connections around a paper, topic or saved view.

Workspaces

HomePick up where your research flow left off.InboxHandle requests, replies and notifications from one place.CollectionsCurate reading lists, evidence sets and shareable dossiers.ResearchSee your private provenance graph, trail and imports.CollaborationMove from discovery into focused discussion rooms.PublishDraft, preview and publish full-text research articles.

Network

ExpertsFind specialists worth following or asking for help.CommunitiesJoin research circles organized around shared work.PartnersSee external organizations active around the same topics.

Opportunities

ListingsBrowse roles, calls and collaborations with clearer fit signals.

Account

Log inReturn to your reading and collaboration workspace.Create accountStart saving work, following people and joining teams.
Log inCreate account

Researcher profile

Darko Stefanovic

Darko Stefanovic contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate
Neural and Evolutionary ComputingEmerging TechnologiesBiological Physicscond-mat.stat-mechcond-mat.dis-nnMachine LearningQuantitative Methods

Trust snapshot

Quick read

Trust 21 - Emerging
13works
0followers
7topics
4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Research graph

See the researcher in context

Open full explorer

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

13 published item(s)

preprint2019arXiv

Tethered single-legged molecular spiders on independent 1D tracks

We study the motion of random walkers with residence time bias between first and subsequent visits to a site, as a model for synthetic molecular walkers composed of coupled DNAzyme legs known as molecular spiders. The mechanism of the transient superdiffusion has been explained via the emergence of a boundary between the new and the previously visited sites, and the tendency of the multi-legged spider to cling to this boundary, provided residence time for a first visit to a site is longer than for subsequent visits. Using both kinetic Monte Carlo simulation and an analytical approach, we model a system that consists of single-legged walkers, each on its own one-dimensional track, connected by a "leash", i.e., a kinematic constraint that no two spiders can be more than a certain distance apart. Even though a single one-legged walker does not at all exhibit directional, superdiffusive motion, we find that a team of one-legged walkers on parallel tracks, connected by a flexible tether, does enjoy a superdiffusive transient. Furthermore, the one-legged walker teams exhibit a greater expected number of steps per boundary period and are able to diffuse more quickly through the product sea than two-legged walkers, which leads to longer periods of superdiffusion.

0 citations0 reviewsNo code linkedOpen access
preprint2016arXiv

Memory and Information Processing in Recurrent Neural Networks

Recurrent neural networks (RNN) are simple dynamical systems whose computational power has been attributed to their short-term memory. Short-term memory of RNNs has been previously studied analytically only for the case of orthogonal networks, and only under annealed approximation, and uncorrelated input. Here for the first time, we present an exact solution to the memory capacity and the task-solving performance as a function of the structure of a given network instance, enabling direct determination of the function--structure relation in RNNs. We calculate the memory capacity for arbitrary networks with exponentially correlated input and further related it to the performance of the system on signal processing tasks in a supervised learning setup. We compute the expected error and the worst-case error bound as a function of the spectra of the network and the correlation structure of its inputs and outputs. Our results give an explanation for learning and generalization of task solving using short-term memory, which is crucial for building alternative computer architectures using physical phenomena based on the short-term memory principle.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Computational Capacity and Energy Consumption of Complex Resistive Switch Networks

Resistive switches are a class of emerging nanoelectronics devices that exhibit a wide variety of switching characteristics closely resembling behaviors of biological synapses. Assembled into random networks, such resistive switches produce emerging behaviors far more complex than that of individual devices. This was previously demonstrated in simulations that exploit information processing within these random networks to solve tasks that require nonlinear computation as well as memory. Physical assemblies of such networks manifest complex spatial structures and basic processing capabilities often related to biologically-inspired computing. We model and simulate random resistive switch networks and analyze their computational capacities. We provide a detailed discussion of the relevant design parameters and establish the link to the physical assemblies by relating the modeling parameters to physical parameters. More globally connected networks and an increased network switching activity are means to increase the computational capacity linearly at the expense of exponentially growing energy consumption. We discuss a new modular approach that exhibits higher computational capacities and energy consumption growing linearly with the number of networks used. The results show how to optimize the trade-off between computational capacity and energy efficiency and are relevant for the design and fabrication of novel computing architectures that harness random assemblies of emerging nanodevices.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Exploring Transfer Function Nonlinearity in Echo State Networks

Supralinear and sublinear pre-synaptic and dendritic integration is considered to be responsible for nonlinear computation power of biological neurons, emphasizing the role of nonlinear integration as opposed to nonlinear output thresholding. How, why, and to what degree the transfer function nonlinearity helps biologically inspired neural network models is not fully understood. Here, we study these questions in the context of echo state networks (ESN). ESN is a simple neural network architecture in which a fixed recurrent network is driven with an input signal, and the output is generated by a readout layer from the measurements of the network states. ESN architecture enjoys efficient training and good performance on certain signal-processing tasks, such as system identification and time series prediction. ESN performance has been analyzed with respect to the connectivity pattern in the network structure and the input bias. However, the effects of the transfer function in the network have not been studied systematically. Here, we use an approach tanh on the Taylor expansion of a frequently used transfer function, the hyperbolic tangent function, to systematically study the effect of increasing nonlinearity of the transfer function on the memory, nonlinear capacity, and signal processing performance of ESN. Interestingly, we find that a quadratic approximation is enough to capture the computational power of ESN with tanh function. The results of this study apply to both software and hardware implementation of ESN.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Hierarchical Composition of Memristive Networks for Real-Time Computing

Advances in materials science have led to physical instantiations of self-assembled networks of memristive devices and demonstrations of their computational capability through reservoir computing. Reservoir computing is an approach that takes advantage of collective system dynamics for real-time computing. A dynamical system, called a reservoir, is excited with a time-varying signal and observations of its states are used to reconstruct a desired output signal. However, such a monolithic assembly limits the computational power due to signal interdependency and the resulting correlated readouts. Here, we introduce an approach that hierarchically composes a set of interconnected memristive networks into a larger reservoir. We use signal amplification and restoration to reduce reservoir state correlation, which improves the feature extraction from the input signals. Using the same number of output signals, such a hierarchical composition of heterogeneous small networks outperforms monolithic memristive networks by at least 20% on waveform generation tasks. On the NARMA-10 task, we reduce the error by up to a factor of 2 compared to homogeneous reservoirs with sigmoidal neurons, whereas single memristive networks are unable to produce the correct result. Hierarchical composition is key for solving more complex tasks with such novel nano-scale hardware.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Product Reservoir Computing: Time-Series Computation with Multiplicative Neurons

Echo state networks (ESN), a type of reservoir computing (RC) architecture, are efficient and accurate artificial neural systems for time series processing and learning. An ESN consists of a core of recurrent neural networks, called a reservoir, with a small number of tunable parameters to generate a high-dimensional representation of an input, and a readout layer which is easily trained using regression to produce a desired output from the reservoir states. Certain computational tasks involve real-time calculation of high-order time correlations, which requires nonlinear transformation either in the reservoir or the readout layer. Traditional ESN employs a reservoir with sigmoid or tanh function neurons. In contrast, some types of biological neurons obey response curves that can be described as a product unit rather than a sum and threshold. Inspired by this class of neurons, we introduce a RC architecture with a reservoir of product nodes for time series computation. We find that the product RC shows many properties of standard ESN such as short-term memory and nonlinear capacity. On standard benchmarks for chaotic prediction tasks, the product RC maintains the performance of a standard nonlinear ESN while being more amenable to mathematical analysis. Our study provides evidence that such networks are powerful in highly nonlinear tasks owing to high-order statistics generated by the recurrent product node reservoir.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

A Comparative Study of Reservoir Computing for Temporal Signal Processing

Reservoir computing (RC) is a novel approach to time series prediction using recurrent neural networks. In RC, an input signal perturbs the intrinsic dynamics of a medium called a reservoir. A readout layer is then trained to reconstruct a target output from the reservoir's state. The multitude of RC architectures and evaluation metrics poses a challenge to both practitioners and theorists who study the task-solving performance and computational power of RC. In addition, in contrast to traditional computation models, the reservoir is a dynamical system in which computation and memory are inseparable, and therefore hard to analyze. Here, we compare echo state networks (ESN), a popular RC architecture, with tapped-delay lines (DL) and nonlinear autoregressive exogenous (NARX) networks, which we use to model systems with limited computation and limited memory respectively. We compare the performance of the three systems while computing three common benchmark time series: H{é}non Map, NARMA10, and NARMA20. We find that the role of the reservoir in the reservoir computing paradigm goes beyond providing a memory of the past inputs. The DL and the NARX network have higher memorization capability, but fall short of the generalization power of the ESN.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

A Model for Variation- and Fault-Tolerant Digital Logic using Self-Assembled Nanowire Architectures

Reconfiguration has been used for both defect- and fault-tolerant nanoscale architectures with regular structure. Recent advances in self-assembled nanowires have opened doors to a new class of electronic devices with irregular structure. For such devices, reservoir computing has been shown to be a viable approach to implement computation. This approach exploits the dynamical properties of a system rather than specifics of its structure. Here, we extend a model of reservoir computing, called the echo state network, to reflect more realistic aspects of self-assembled nanowire networks. As a proof of concept, we use echo state networks to implement basic building blocks of digital computing: AND, OR, and XOR gates, and 2-bit adder and multiplier circuits. We show that the system can operate perfectly in the presence of variations five orders of magnitude higher than ITRS's 2005 target, $\bm{6\%}$, and achieves success rates $\bm{6}$ times higher than related approaches at half the cost. We also describe an adaptive algorithm that can detect faults in the system and reconfigure it to resume perfect operational condition.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Lattice gases with a point source

We study diffusive lattice gases with local injection of particles, namely we assume that whenever the origin becomes empty, a new particle is immediately injected into the origin. We consider two lattice gases: a symmetric simple exclusion process and random walkers. The interplay between the injection events and the positions of the particles already present implies an effective collective interaction even for the ostensibly non-interacting random walkers. We determine the average total number of particles entering into the initially empty system. We also compute the average total number of distinct sites visited by all particles, and discuss the shape of the visited domain and the statistics of visits.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Reservoir Computing Approach to Robust Computation using Unreliable Nanoscale Networks

As we approach the physical limits of CMOS technology, advances in materials science and nanotechnology are making available a variety of unconventional computing substrates that can potentially replace top-down-designed silicon-based computing devices. Inherent stochasticity in the fabrication process and nanometer scale of these substrates inevitably lead to design variations, defects, faults, and noise in the resulting devices. A key challenge is how to harness such devices to perform robust computation. We propose reservoir computing as a solution. In reservoir computing, computation takes place by translating the dynamics of an excited medium, called a reservoir, into a desired output. This approach eliminates the need for external control and redundancy, and the programming is done using a closed-form regression problem on the output, which also allows concurrent programming using a single device. Using a theoretical model, we show that both regular and irregular reservoirs are intrinsically robust to structural noise as they perform computation.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Towards a Calculus of Echo State Networks

Reservoir computing is a recent trend in neural networks which uses the dynamical perturbations on the phase space of a system to compute a desired target function. We present how one can formulate an expectation of system performance in a simple class of reservoir computing called echo state networks. In contrast with previous theoretical frameworks, which only reveal an upper bound on the total memory in the system, we analytically calculate the entire memory curve as a function of the structure of the system and the properties of the input and the target function. We demonstrate the precision of our framework by validating its result for a wide range of system sizes and spectral radii. Our analytical calculation agrees with numerical simulations. To the best of our knowledge this work presents the first exact analytical characterization of the memory curve in echo state networks.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

First Passage Properties of Molecular Spiders

Molecular spiders are synthetic catalytic DNA-based nanoscale walkers. We study the mean first passage time for abstract models of spiders moving on a finite two-dimensional lattice with various boundary conditions, and compare it with the mean first passage time of spiders moving on a one-dimensional track. We evaluate by how much the slowdown on newly visited sites, owing to catalysis, can improve the mean first passage time of spiders and show that in one dimension, when both ends of the track are an absorbing boundary, the performance gain is lower than in two dimensions, when the absorbing boundary is a circle; this persists even when the absorbing boundary is a single site.

0 citations0 reviewsNo code linkedOpen access
preprint2013arXiv

Superdiffusive transport by multivalent molecular walkers moving under load

We introduce a model for translational molecular motors to demonstrate that a multivalent catalytic walker with flexible, uncoordinated legs can transform the free energy of surface-bound substrate sites into mechanical work and undergo biased, superdiffusive motion, even in opposition to an external load force. The walker in the model lacks any inherent orientation of body or track, and its legs have no chemomechanical coupling other than the passive constraint imposed by their connection to a common body. Yet, under appropriate kinetic conditions the walker's motion is biased in the direction of unvisited sites, which allows the walker to move nearly ballistically away from the origin as long as a local supply of unmodified substrate sites is available. The multivalent random walker model is mathematically formulated as a continuous-time Markov process and is studied numerically. We use Monte Carlo simulations to generate ensemble estimates of the mean squared displacement and mean work done for this non-ergodic system. Our results show that a residence time bias between visited and unvisited sites leads to superdiffusive motion over significant times and distances. This mechanism can be used to adapt any enzyme--substrate system with appropriate kinetics for use as a functional chemical implementation of a molecular motor, without the need for structural anisotropy or conformationally mediated chemomechanical coordination.

0 citations0 reviewsNo code linkedOpen access