Matthew Kay Fei Lee
Skip Abstract Section
Abstract
Advances in non-volatile resistive switching random access memory (RRAM) have made it a promising memory technology with potential applications in low-power and embedded in-memory computing devices owing to a number of advantages such as low-energy consumption, low area cost and good scaling. There have been proposals to employ RRAM in architecting chips for neuromorphic computing and artificial neural networks where matrix-vector multiplication can be computed in the analog domain in a single timestep. However, it is challenging to employ RRAM devices in neuromorphic chips owing to the non-ideal behavior of RRAM. In this article, we propose a cycle-accurate and scalable system-level simulator that can be used to study the effects of using RRAM devices in neuromorphic computing chips. The simulator models a spatial neuromorphic chip architecture containing many neural cores with RRAM crossbars connected via a Network-on-Chip (NoC). We focus on system-level simulation and demonstrate the effectiveness of our simulator in understanding how non-linear RRAM effects such as stuck-at-faults (SAFs), write variability, and random telegraph noise (RTN) can impact an application’s behavior. By using our simulator, we show that RTN and write variability can have adverse effects on an application. Nevertheless, we show that these effects can be mitigated through proper design choices and the implementation of a write-verify scheme.
- Filipp Akopyan, Jun Sawada, Andrew Cassidy, Rodrigo Alvarez-Icaza, John Arthur, Paul Merolla, Nabil Imam, Yutaka Nakamura, Pallab Datta, Gi-Joon Nam, et al. 2015. TrueNorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 34, 10 (2015), 1537--1557.Google Scholar
Digital Library
- J. Arthur and K. Boahen. 2004. Recurrently connected silicon neurons with active dendrites for one-shot learning. In Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN’04), Vol. 3. IEEE, 1699--1704.Google Scholar
- Bhaswar Chakrabarti, Miguel Angel Lastras-Montaño, Gina Adam, Mirko Prezioso, Brian Hoskins, M. Payvand, A. Madhavan, A. Ghofrani, L. Theogarajan, K.-T. Cheng, et al. 2017. A multiply-add engine with monolithically integrated 3D memristor crossbar/CMOS hybrid circuit. Scientific Reports 7 (2017), 42429.Google ScholarCross Ref
- A. Ankit et al. 2017. RESPARC: A reconfigurable and energy-efficient architecture with memristive crossbars for deep spiking neural networks. In Proceedings of the 54th Annual Design Automation Conference 2017. ACM, 27:1--27:6. Google ScholarDigital Library
- A. Davison et al. 2009. PyNN: A common interface for neuronal network simulators. Frontiers in Neuroinformatics 2 (2009), 11.Google Scholar
- A. Shafiee et al. 2016. ISAAC: A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA). 14--26. Google ScholarDigital Library
- B. Benjamin et al. 2014. Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations. Proceedings of the IEEE 102, 5 (2014), 699--716.Google ScholarCross Ref
- D. Yudanov et al. 2010. GPU-based simulation of spiking neural networks with real-time performance and high accuracy. In International Joint Conference on Neural Networks (IJCNN’10). IEEE, 1--8.Google ScholarCross Ref
- E. Painkras et al. 2013. SpiNNaker: A 1-W 18-core system-on-chip for massively-parallel neural network simulation. IEEE Journal of Solid-State Circuits 48, 8 (Aug. 2013), 1943--1953.Google ScholarCross Ref
- G. Indiveri et al. 2013. Integration of nanoscale memristor synapses in neuromorphic computing architectures. Nanotechnology 24, 38 (2013), 384010.Google ScholarCross Ref
- H. Wong et al. 2012. Metal-oxide RRAM. Proceedings of the IEEE 100, 6 (June 2012), 1951--1970.Google ScholarCross Ref
- J. Schemmel et al. 2010. A wafer-scale neuromorphic hardware system for large-scale neural modeling. In Proceedings of 2010 IEEE International Symposium on Circuits and Systems (ISCAS’10). IEEE, 1947--1950.Google ScholarCross Ref
- K. Minkovich et al. 2014. HRLSim: A high performance spiking neural network simulator for GPGPU clusters. IEEE Transactions on Neural Networks and Learning Systems 25, 2 (2014), 316--331.Google ScholarCross Ref
- L. Xia et al. 2018. MNSIM: Simulation platform for memristor-based neuromorphic computing system. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 37, 5 (May 2018), 1009--1022.Google Scholar
- M. Hu et al. 2016. Dot-product engine for neuromorphic computing: Programming 1T1M crossbar to accelerate matrix-vector multiplication. In 53rd ACM/EDAC/IEEE Design Automation Conference (DAC’16). 1--6. Google ScholarDigital Library
- M. Prezioso et al. 2015. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 7550 (2015), 61--64.Google Scholar
- P. Merolla et al. 2014. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345, 6197 (2014), 668--673.Google Scholar
- P. U. Diehl et al. 2015. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In International Joint Conference on Neural Networks (IJCNN’15). 1--8.Google ScholarCross Ref
- P. Yao et al. 2017. Face classification using electronic synapses. Nature Communications 8 (2017), 15199.Google ScholarCross Ref
- S. Ambrogio et al. 2014. Statistical fluctuations in HfOx resistive-switching memory: Part I - Set/reset variability. IEEE Transactions on Electron Devices 61, 8 (Aug 2014), 2912--2919.Google Scholar
- S. Ambrogio et al. 2014. Statistical fluctuations in HfOx resistive-switching memory: Part II - Random telegraph noise. IEEE Transactions on Electron Devices 61, 8 (Aug 2014), 2920--2927.Google Scholar
- S. Esser et al. 2016. Convolutional networks for fast, energy-efficient neuromorphic computing. Proceedings of the National Academy of Sciences 113, 41 (2016), 11441--11446.Google ScholarCross Ref
- S. H. Jo et al. 2010. Nanoscale memristor device as synapse in neuromorphic systems. Nano Letters 10, 4 (2010), 1297--1301.Google ScholarCross Ref
- S. Narayanan et al. 2017. INXS: Bridging the throughput and energy gap for spiking neural networks. In International Joint Conference on Neural Networks (IJCNN’17). IEEE, 2451--2459.Google ScholarCross Ref
- S. Park et al. 2013. Neuromorphic speech systems using advanced ReRAM-based synapse. In IEEE International Electron Devices Meeting (IEDM’13). IEEE, 25--6.Google ScholarCross Ref
- T. Tang et al. 2015. Spiking neural network with RRAM: Can we use it for real-world application? In Proceedings of the Design, Automation 8 Test in Europe Conference 8 Exhibition. EDA Consortium, 860--865. Google ScholarDigital Library
- A. Fidjeland and M. Shanahan. 2010. Accelerated simulation of spiking neural networks using GPUs. In International Joint Conference on Neural Networks (IJCNN’10). IEEE, 1--8.Google Scholar
- M. Gewaltig and M. Diesmann. 2007. NEST (NEural simulation tool). Scholarpedia 2, 4 (2007), 1430.Google ScholarCross Ref
- Amirali Ghofrani, Siddharth Gaba, Melika Payvand, Wei Lu, Luke Theogarajan, Kwang-Ting Cheng, et al. 2015. A low-power variation-aware adaptive write scheme for access-transistor-free memristive memory. ACM Journal on Emerging Technologies in Computing Systems (JETC) 12, 1 (2015), 3. Google ScholarDigital Library
- D. Goodman and R. Brette. 2009. The brain simulator. Frontiers in Neuroscience 3 (2009), 26.Google ScholarCross Ref
- Miao Hu, John Paul Strachan, Zhiyong Li, R. Stanley, et al. 2016. Dot-product engine as computing memory to accelerate machine learning algorithms. In 17th International Symposium on Quality Electronic Design (ISQED’16). IEEE, 374--379.Google ScholarCross Ref
- E. M. Izhikevich. 2003. Simple model of spiking neurons. IEEE Transactions on Neural Networks 14, 6 (Nov 2003), 1569--1572. Google ScholarDigital Library
- Yu Ji, Youhui Zhang, Wenguang Chen, and Yuan Xie. 2018. Bridge the gap between neural networks and neuromorphic hardware with a neural network compiler. In Proceedings of the 23rd International Conference on Architectural Support for Programming Languages and Operating Systems. ACM, 448--460. Google ScholarDigital Library
- Yu Ji, Youhui Zhang, He Liu, and Weimin Zheng. 2016. Optimized mapping spiking neural networks onto network-on-chip. In International Conference on Algorithms and Architectures for Parallel Processing. Springer, 38--52.Google ScholarCross Ref
- Andrew B. Kahng, Bin Li, Li-Shiuan Peh, and Kambiz Samadi. 2009. ORION 2.0: A fast and accurate NoC power and area model for early-stage design space exploration. In Proceedings of the Conference on Design, Automation and Test in Europe. European Design and Automation Association, 423--428. Google ScholarDigital Library
- V. G. Karpov and D. Niraula. 2017. Log-normal statistics in filamentary RRAM devices and related systems. IEEE Electron Device Letters 38, 9 (2017), 1240--1243.Google ScholarCross Ref
- Boxun Li, Lixue Xia, Peng Gu, Yu Wang, and Huazhong Yang. 2015. Merging the interface: Power, area and accuracy co-optimization for RRAM crossbar-based mixed-signal computing system. In Proceedings of the 52nd Annual Design Automation Conference. ACM, 13. Google ScholarDigital Library
- Ye Li, Bertan Bakkaloglu, and Chaitali Chakrabarti. 2007. A system level energy model and energy-quality evaluation for integrated transceiver front-ends. IEEE Transactions on Very Large Scale Integration (VLSI) Systems 15, 1 (2007), 90--103. Google ScholarDigital Library
- W. Maass. 1997. Networks of spiking neurons: The third generation of neural network models. Neural Networks 10, 9 (1997), 1659--1671. Google ScholarCross Ref
- Don Monroe. 2014. Neuromorphic computing gets ready for the (really) big time. Communications of the ACM 57, 6 (2014), 13--15. Google ScholarDigital Library
- Hiroki Noguchi, Kazutaka Ikegami, Satoshi Takaya, Eishi Arima, Keiichi Kushida, Atsushi Kawasumi, Hiroyuki Hara, Keiko Abe, Naoharu Shimomura, Junichi Ito, et al. 2016. 7.2 4MB STT-MRAM-based cache with memory-access-aware power optimization and write-verify-write/read-modify-write scheme. In IEEE International Solid-State Circuits Conference (ISSCC’16). IEEE, 132--133.Google ScholarCross Ref
- M. Papamichael and J. Hoe. 2012. CONNECT: Re-examining conventional wisdom for designing NoCs in the context of FPGAs. In Proceedings of the ACM/SIGDA International Symposium on Field Programmable Gate Arrays. 37--46. Google ScholarDigital Library
- R. Preisslet al. 2012. COMPASS: A scalable simulator for an architecture for cognitive computing. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis. 54. Google ScholarDigital Library
- D. Querlioz, O. Bichler, and C. Gamrat. 2011. Simulation of a memristor-based spiking neural network immune to device variations. In International Joint Conference on Neural Networks (IJCNN’11). IEEE, 1775--1781.Google Scholar
- K. Zaghloul and K. Boahen. 2004. Optic nerve signals in a neuromorphic chip I: Outer and inner retina models. IEEE Transactions on Biomedical Engineering 51, 4 (2004), 657--666.Google ScholarCross Ref
Index Terms
- A System-Level Simulator for RRAM-Based Neuromorphic Computing Chips
Recommendations
Research on feasibility of using a Transient Voltage Suppressor as the selection device for bipolar RRAM
In this paper, the Transient Voltage Suppressor (TVS) is provided as the selection device for bipolar RRAM crossbar array applications. Simulation results suggest that, by using the TVS as a selection device, the sneak current in the crossbar array can ...
Oxide based two diodes-one resistor structure for bipolar RRAM crossbar array
Graphical abstractI-V characteristics of the anti-parallel connected Ag/TiOx/Ti diodes selector in series with the Cu/HfO2/Pt bipolar RRAM cell.Display Omitted A new bipolar selector based on anti-parallel connected diodes is proposed.Anti-parallel ...
An RRAM-based MLC design approach
RRAM is an emerging technology with vast applications in computation and storage. RRAM with MLC capabilities is more interesting owing to its higher storage density. In this study, we proposed an approach for RRAM-based MLC design. The approach starts ...
Comments