BZPEERPapers, people, evidence and research paths
Menu

Discover

SearchFind papers, people, topics, institutions and opportunities from one place.GraphSee how papers, authors, topics and evidence connect.

Research tools

HomeContinue from the papers, people and topics that matter now.CollectionsTurn saved papers into reading lists, evidence maps and shareable context.ResearchReconstruct the path that led you from search to paper to question.CollaborationMove from a paper, topic or expert request into a focused thread.PublishCreate full-text research articles with metadata, formulas and versions.

Network

ExpertsFind specialists whose work and reviews explain why they can help.CommunitiesJoin research groups organized around papers, topics and shared evidence.

Opportunities

ListingsBrowse roles, calls and collaborations connected to real research context.

Account

Log inReturn to your saved papers, graph views and active threads.Create accountStart saving papers, building a research trail and joining teams.
Log inCreate account

Source author record

Bipin Rajendran

Bipin Rajendran appears in the imported research catalog. Authorship, coauthor and topic links are available while profile ownership is still unclaimed.

ResearcherUnclaimed source record
Emerging TechnologiesNeural and Evolutionary ComputingMachine Learningeess.SPHardware ArchitectureNeurons and CognitionArtificial Intelligencecond-mat.mes-hallcond-mat.mtrl-sci

Catalog footprint

What is connected

13works
9topics
4close collaborators

Actions

Connect this record

Log in to claim
Open graphBrowse works

Research graph

See the researcher in context

Open full explorer

Inspect adjacent papers, topics, institutions and collaborators without losing the researcher page.

Building this map preview

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

13 published item(s)

preprint2021arXiv

Fast On-Device Adaptation for Spiking Neural Networks via Online-Within-Online Meta-Learning

Spiking Neural Networks (SNNs) have recently gained popularity as machine learning models for on-device edge intelligence for applications such as mobile healthcare management and natural language processing due to their low power profile. In such highly personalized use cases, it is important for the model to be able to adapt to the unique features of an individual with only a minimal amount of training data. Meta-learning has been proposed as a way to train models that are geared towards quick adaptation to new tasks. The few existing meta-learning solutions for SNNs operate offline and require some form of backpropagation that is incompatible with the current neuromorphic edge-devices. In this paper, we propose an online-within-online meta-learning rule for SNNs termed OWOML-SNN, that enables lifelong learning on a stream of tasks, and relies on local, backprop-free, nested updates.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Hybrid In-memory Computing Architecture for the Training of Deep Neural Networks

The cost involved in training deep neural networks (DNNs) on von-Neumann architectures has motivated the development of novel solutions for efficient DNN training accelerators. We propose a hybrid in-memory computing (HIC) architecture for the training of DNNs on hardware accelerators that results in memory-efficient inference and outperforms baseline software accuracy in benchmark tasks. We introduce a weight representation technique that exploits both binary and multi-level phase-change memory (PCM) devices, and this leads to a memory-efficient inference accelerator. Unlike previous in-memory computing-based implementations, we use a low precision weight update accumulator that results in more memory savings. We trained the ResNet-32 network to classify CIFAR-10 images using HIC. For a comparable model size, HIC-based training outperforms baseline network, trained in floating-point 32-bit (FP32) precision, by leveraging appropriate network width multiplier. Furthermore, we observe that HIC-based training results in about 50% less inference model size to achieve baseline comparable accuracy. We also show that the temporal drift in PCM devices has a negligible effect on post-training inference accuracy for extended periods (year). Finally, our simulations indicate HIC-based training naturally ensures that the number of write-erase cycles seen by the devices is a small fraction of the endurance limit of PCM, demonstrating the feasibility of this architecture for achieving hardware platforms that can learn in the field.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Accurate deep neural network inference using computational phase-change memory

In-memory computing is a promising non-von Neumann approach for making energy-efficient deep learning inference hardware. Crossbar arrays of resistive memory devices can be used to encode the network weights and perform efficient analog matrix-vector multiplications without intermediate movements of data. However, due to device variability and noise, the network needs to be trained in a specific way so that transferring the digitally trained weights to the analog resistive memory devices will not result in significant loss of accuracy. Here, we introduce a methodology to train ResNet-type convolutional neural networks that results in no appreciable accuracy loss when transferring weights to in-memory computing hardware based on phase-change memory (PCM). We also propose a compensation technique that exploits the batch normalization parameters to improve the accuracy retention over time. We achieve a classification accuracy of 93.7% on the CIFAR-10 dataset and a top-1 accuracy on the ImageNet benchmark of 71.6% after mapping the trained weights to PCM. Our hardware results on CIFAR-10 with ResNet-32 demonstrate an accuracy above 93.5% retained over a one day period, where each of the 361,722 synaptic weights of the network is programmed on just two PCM devices organized in a differential configuration.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Bio-mimetic Synaptic Plasticity and Learning in a sub-500mV Cu/SiO$_2$/W Memristor

The computational efficiency of the human brain is believed to stem from the parallel information processing capability of neurons with integrated storage in synaptic interconnections programmed by local spike triggered learning rules such as spike timing dependent plasticity (STDP). The extremely low operating voltages (approximately $100\,$mV) used to trigger neuronal signaling and synaptic adaptation is believed to be a critical reason for the brain's power efficiency. We demonstrate the feasibility of spike triggered STDP behavior in a two-terminal Cu/SiO$_2$/W memristive device capable of operating below $500\,$mV. We analyze the state-dependent nature of conductance updates in the device to develop a phenomenological model. Using the model, we evaluate the potential of such devices to generate precise spike times under supervised learning conditions and classify handwritten digits from the MNIST dataset in an unsupervised learning setting. The results form a promising step towards creating a low power synaptic device capable of on-chip learning.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

ESSOP: Efficient and Scalable Stochastic Outer Product Architecture for Deep Learning

Deep neural networks (DNNs) have surpassed human-level accuracy in a variety of cognitive tasks but at the cost of significant memory/time requirements in DNN training. This limits their deployment in energy and memory limited applications that require real-time learning. Matrix-vector multiplications (MVM) and vector-vector outer product (VVOP) are the two most expensive operations associated with the training of DNNs. Strategies to improve the efficiency of MVM computation in hardware have been demonstrated with minimal impact on training accuracy. However, the VVOP computation remains a relatively less explored bottleneck even with the aforementioned strategies. Stochastic computing (SC) has been proposed to improve the efficiency of VVOP computation but on relatively shallow networks with bounded activation functions and floating-point (FP) scaling of activation gradients. In this paper, we propose ESSOP, an efficient and scalable stochastic outer product architecture based on the SC paradigm. We introduce efficient techniques to generalize SC for weight update computation in DNNs with the unbounded activation functions (e.g., ReLU), required by many state-of-the-art networks. Our architecture reduces the computational cost by re-using random numbers and replacing certain FP multiplication operations by bit shift scaling. We show that the ResNet-32 network with 33 convolution layers and a fully-connected layer can be trained with ESSOP on the CIFAR-10 dataset to achieve baseline comparable accuracy. Hardware design of ESSOP at 14nm technology node shows that, compared to a highly pipelined FP16 multiplier design, ESSOP is 82.2% and 93.7% better in energy and area efficiency respectively for outer product computation.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Memristors -- from In-memory computing, Deep Learning Acceleration, Spiking Neural Networks, to the Future of Neuromorphic and Bio-inspired Computing

Machine learning, particularly in the form of deep learning, has driven most of the recent fundamental developments in artificial intelligence. Deep learning is based on computational models that are, to a certain extent, bio-inspired, as they rely on networks of connected simple computing units operating in parallel. Deep learning has been successfully applied in areas such as object/pattern recognition, speech and natural language processing, self-driving vehicles, intelligent self-diagnostics tools, autonomous robots, knowledgeable personal assistants, and monitoring. These successes have been mostly supported by three factors: availability of vast amounts of data, continuous growth in computing power, and algorithmic innovations. The approaching demise of Moore's law, and the consequent expected modest improvements in computing power that can be achieved by scaling, raise the question of whether the described progress will be slowed or halted due to hardware limitations. This paper reviews the case for a novel beyond CMOS hardware technology, memristors, as a potential solution for the implementation of power-efficient in-memory computing, deep learning accelerators, and spiking neural networks. Central themes are the reliance on non-von-Neumann computing architectures and the need for developing tailored learning and inference algorithms. To argue that lessons from biology can be useful in providing directions for further progress in artificial intelligence, we briefly discuss an example based reservoir computing. We conclude the review by speculating on the big picture view of future neuromorphic and brain-inspired computing systems.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

Mixed-precision deep learning based on computational memory

Deep neural networks (DNNs) have revolutionized the field of artificial intelligence and have achieved unprecedented success in cognitive tasks such as image and speech recognition. Training of large DNNs, however, is computationally intensive and this has motivated the search for novel computing architectures targeting this application. A computational memory unit with nanoscale resistive memory devices organized in crossbar arrays could store the synaptic weights in their conductance states and perform the expensive weighted summations in place in a non-von Neumann manner. However, updating the conductance states in a reliable manner during the weight update process is a fundamental challenge that limits the training accuracy of such an implementation. Here, we propose a mixed-precision architecture that combines a computational memory unit performing the weighted summations and imprecise conductance updates with a digital processing unit that accumulates the weight updates in high precision. A combined hardware/software training experiment of a multilayer perceptron based on the proposed architecture using a phase-change memory (PCM) array achieves 97.73% test accuracy on the task of classifying handwritten digits (based on the MNIST dataset), within 0.6% of the software baseline. The architecture is further evaluated using accurate behavioral models of PCM on a wide class of networks, namely convolutional neural networks, long-short-term-memory networks, and generative-adversarial networks. Accuracies comparable to those of floating-point implementations are achieved without being constrained by the non-idealities associated with the PCM devices. A system-level study demonstrates 173x improvement in energy efficiency of the architecture when used for training a multilayer perceptron compared with a dedicated fully digital 32-bit implementation.

0 citations0 reviewsNo code linkedOpen access
preprint2020arXiv

SpinAPS: A High-Performance Spintronic Accelerator for Probabilistic Spiking Neural Networks

We discuss a high-performance and high-throughput hardware accelerator for probabilistic Spiking Neural Networks (SNNs) based on Generalized Linear Model (GLM) neurons, that uses binary STT-RAM devices as synapses and digital CMOS logic for neurons. The inference accelerator, termed "SpinAPS" for Spintronic Accelerator for Probabilistic SNNs, implements a principled direct learning rule for first-to-spike decoding without the need for conversion from pre-trained ANNs. The proposed solution is shown to achieve comparable performance with an equivalent ANN on handwritten digit and human activity recognition benchmarks. The inference engine, SpinAPS, is shown through software emulation tools to achieve 4x performance improvement in terms of GSOPS/W/mm2 when compared to an equivalent SRAM-based design. The architecture leverages probabilistic spiking neural networks that employ first-to-spike decoding rule to make inference decisions at low latencies, achieving 75% of the test performance in as few as 4 algorithmic time steps on the handwritten digit benchmark. The accelerator also exhibits competitive performance with other memristor-based DNN/SNN accelerators and state-of-the-art GPUs.

0 citations0 reviewsNo code linkedOpen access
preprint2019arXiv

Learning First-to-Spike Policies for Neuromorphic Control Using Policy Gradients

Artificial Neural Networks (ANNs) are currently being used as function approximators in many state-of-the-art Reinforcement Learning (RL) algorithms. Spiking Neural Networks (SNNs) have been shown to drastically reduce the energy consumption of ANNs by encoding information in sparse temporal binary spike streams, hence emulating the communication mechanism of biological neurons. Due to their low energy consumption, SNNs are considered to be important candidates as co-processors to be implemented in mobile devices. In this work, the use of SNNs as stochastic policies is explored under an energy-efficient first-to-spike action rule, whereby the action taken by the RL agent is determined by the occurrence of the first spike among the output neurons. A policy gradient-based algorithm is derived considering a Generalized Linear Model (GLM) for spiking neurons. Experimental results demonstrate the capability of online trained SNNs as stochastic policies to gracefully trade energy consumption, as measured by the number of spikes, and control performance. Significant gains are shown as compared to the standard approach of converting an offline trained ANN into an SNN.

0 citations0 reviewsNo code linkedOpen access
preprint2015arXiv

Programming current reduction via enhanced asymmetry-induced thermoelectric effects in vertical nanopillar phase change memory cells

Thermoelectric effects are envisioned to reduce programming currents in nanopillar phase change memory cells. However, due to the inherent symmetry in such a structure, the contribution due to thermoelectric effects on programming currents is minimal. In this work, we propose a hybrid phase change memory structure which incorporates a two-fold asymmetry specifically aimed to favorably enhance thermoelectric effects. The first asymmetry is introduced via an interface layer of low thermal conductivity and high negative Seebeck coefficient, such as, polycrystalline SiGe, between the bottom electrode contact and the active region comprising the phase change material. This results in an enhanced Peltier heating of the active material. The second one is introduced structurally via a taper that results in an angle dependent Thomson heating within the active region. Various device geometries are analyzed using 2D-axis-symmetric simulations to predict the effect on programming currents as well as for different thicknesses of the interface layer. A programming current reduction of up to $60\%$ is predicted for specific cell geometries. Remarkably, we find that due to an interplay of Thomson cooling in the electrode and the asymmetric heating profile inside the active region, the predicted programming current reduction is resilient to fabrication variability.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

A neural circuit for navigation inspired by C. elegans Chemotaxis

We develop an artificial neural circuit for contour tracking and navigation inspired by the chemotaxis of the nematode Caenorhabditis elegans. In order to harness the computational advantages spiking neural networks promise over their non-spiking counterparts, we develop a network comprising 7-spiking neurons with non-plastic synapses which we show is extremely robust in tracking a range of concentrations. Our worm uses information regarding local temporal gradients in sodium chloride concentration to decide the instantaneous path for foraging, exploration and tracking. A key neuron pair in the C. elegans chemotaxis network is the ASEL & ASER neuron pair, which capture the gradient of concentration sensed by the worm in their graded membrane potentials. The primary sensory neurons for our network are a pair of artificial spiking neurons that function as gradient detectors whose design is adapted from a computational model of the ASE neuron pair in C. elegans. Simulations show that our worm is able to detect the set-point with approximately four times higher probability than the optimal memoryless Levy foraging model. We also show that our spiking neural network is much more efficient and noise-resilient while navigating and tracking a contour, as compared to an equivalent non-spiking network. We demonstrate that our model is extremely robust to noise and with slight modifications can be used for other practical applications such as obstacle avoidance. Our network model could also be extended for use in three-dimensional contour tracking or obstacle avoidance.

0 citations0 reviewsNo code linkedOpen access
preprint2014arXiv

Sub-threshold CMOS Spiking Neuron Circuit Design for Navigation Inspired by C. elegans Chemotaxis

We demonstrate a spiking neural network for navigation motivated by the chemotaxis network of Caenorhabditis elegans. Our network uses information regarding temporal gradients in the tracking variable's concentration to make navigational decisions. The gradient information is determined by mimicking the underlying mechanisms of the ASE neurons of C. elegans. Simulations show that our model is able to forage and track a target set-point in extremely noisy environments. We develop a VLSI implementation for the main gradient detector neurons, which could be integrated with standard comparator circuitry to develop a robust circuit for navigation and contour tracking.

0 citations0 reviewsNo code linkedOpen access
preprint2010arXiv

Phase change memory technology

We survey the current state of phase change memory (PCM), a non-volatile solid-state memory technology built around the large electrical contrast between the highly-resistive amorphous and highly-conductive crystalline states in so-called phase change materials. PCM technology has made rapid progress in a short time, having passed older technologies in terms of both sophisticated demonstrations of scaling to small device dimensions, as well as integrated large-array demonstrators with impressive retention, endurance, performance and yield characteristics. We introduce the physics behind PCM technology, assess how its characteristics match up with various potential applications across the memory-storage hierarchy, and discuss its strengths including scalability and rapid switching speed. We then address challenges for the technology, including the design of PCM cells for low RESET current, the need to control device-to-device variability, and undesirable changes in the phase change material that can be induced by the fabrication procedure. We then turn to issues related to operation of PCM devices, including retention, device-to-device thermal crosstalk, endurance, and bias-polarity effects. Several factors that can be expected to enhance PCM in the future are addressed, including Multi-Level Cell technology for PCM (which offers higher density through the use of intermediate resistance states), the role of coding, and possible routes to an ultra-high density PCM technology.

0 citations0 reviewsNo code linkedOpen access