Abdullah M. Zyarah
Dhireesha Kudithipudi
Skip Abstract Section
Abstract
On-device intelligence is gaining significant attention recently as it offers local data processing and low power consumption. In this research, an on-device training circuitry for threshold-current memristors integrated in a crossbar structure is proposed. Furthermore, alternate approaches of mapping the synaptic weights into fully trained and semi-trained crossbars are investigated. In a semi-trained crossbar, a confined subset of memristors are tuned and the remaining subset of memristors are not programmed. This translates to optimal resource utilization and power consumption, compared to a fully programmed crossbar. The semi-trained crossbar architecture is applicable to a broad class of neural networks. System level verification is performed with an extreme learning machine for binomial and multinomial classification. The total power for a single 4 × 4 layer network, when implemented in IBM 65nm node, is estimated to be ≈42.16μW and the area is estimated to be 26.48μm × 22.35μm.
- Fabien Alibart, Elham Zamanidoost, and Dmitri B. Strukov. 2013. Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nature Commun. 4 (June 2013), 2072.Google Scholar
- Joshua E. Auerbach, Chrisantha Fernando, and Dario Floreano. 2014. Online extreme evolutionary learning machines. In Proceedings of the 14th International Conference on the Synthesis and Simulation of Living Systems: Artificial Life 14. The MIT Press, 465--472.Google ScholarCross Ref
- Julien Borghetti, Gregory S. Snider, Philip J. Kuekes, J. Joshua Yang, Duncan R. Stewart, and R. Stanley Williams. 2010. Memristive switches enable stateful logic operations via material implication. Nature 464, 7290 (2010), 873--876.Google Scholar
- Gangotree Chakma, Md. Musabbir Adnan, Austin R. Wyer, Ryan Weiss, Catherine D. Schuman, and Garrett S. Rose. 2018. Memristive mixed-signal neuromorphic systems: Energy-efficient learning at the circuit-level. IEEE J. Emerg. Select. Topics Circ. Syst. 8, 1 (2018), 125--136.Google ScholarCross Ref
- Leon Chua. 1971. Memristor-the missing circuit element. IEEE Trans. Circ. Theory 18, 5 (1971), 507--519.Google ScholarCross Ref
- Deliang Fan, Mrigank Sharad, and Kaushik Roy. 2014. Design and synthesis of ultralow energy spin-memristor threshold logic. IEEE Trans. Nanotechnol. 13, 3 (2014), 574--583. Google ScholarDigital Library
- Md Raqibul Hasan. 2016. Memristor Based Low Power High Throughput Circuits and Systems Design. Ph.D. Dissertation. University of Dayton.Google Scholar
- Raqibul Hasan, Tarek M. Taha, and Chris Yakopcic. 2017. On-chip training of memristor crossbar-based multi-layer neural networks. Microelectron. J. 66 (2017), 31--40. Google ScholarDigital Library
- Miao Hu, Hai Li, Yiran Chen, Qing Wu, Garrett S. Rose, and Richard W. Linderman. 2014. Memristor crossbar-based neuromorphic computing system: A case study. IEEE Trans. Neural Netw. Learn. Syst. 25, 10 (2014), 1864--1878.Google ScholarCross Ref
- Guang-Bin Huang. 2014. An insight into extreme learning machines: Random neurons, random features and kernels. Cogn. Comput. 6, 3 (2014), 376--390.Google ScholarCross Ref
- Guang-Bin Huang, Hongming Zhou, Xiaojian Ding, and Rui Zhang. 2012. Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst., Man, Cybernet., Part B (Cybernet.) 42, 2 (2012), 513--529. Google ScholarDigital Library
- Guang-Bin Huang, Qin-Yu Zhu, and Chee-Kheong Siew. 2004. Extreme learning machine: A new learning scheme of feedforward neural networks. In Proceedings of the IEEE International Joint Conference on Neural Networks, Vol. 2. IEEE, 985--990.Google Scholar
- Guang-Bin Huang, Qin-Yu Zhu, and Chee-Kheong Siew. 2006. Extreme learning machine: Theory and applications. Neurocomputing 70, 1 (2006), 489--501.Google ScholarCross Ref
- Giacomo Indiveri and Shih-Chii Liu. 2015. Memory and information processing in neuromorphic systems. Proc. IEEE 103, 8 (2015), 1379--1397.Google ScholarCross Ref
- Robert A. Jacobs. 1988. Increased rates of convergence through learning rate adaptation. Neural Netw. 1, 4 (1988), 295--307.Google ScholarCross Ref
- Sung Hyun Jo, Ting Chang, Idongesit Ebong, Bhavitavya B. Bhadviya, Pinaki Mazumder, and Wei Lu. 2010. Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 4 (2010), 1297--1301.Google ScholarCross Ref
- Liyanaarachchi Lekamalage Chamara Kasun, Hongming Zhou, Guang-Bin Huang, and Chi Man Vong. 2013. Representational learning with ELMs for big data. IEEE Intell. Syst. 28, 6 (2013), 31--34.Google Scholar
- Takayuki Kawahara, Riichiro Takemura, Katsuya Miura, Jun Hayakawa, Shoji Ikeda, Young Min Lee, Ryutaro Sasaki, Yasushi Goto, Kenchi Ito, Toshiyasu Meguro, et al. 2008. 2 Mb SPRAM (spin-transfer torque RAM) with bit-by-bit bi-directional current write and parallelizing-direction current read. IEEE J. Solid-State Circ. 43, 1 (2008), 109--120.Google ScholarCross Ref
- Hyongsuk Kim, Maheshwar Pd. Sah, Changju Yang, Tamás Roska, and Leon O. Chua. 2012. Neural synaptic weighting with a pulse-based memristor circuit. IEEE Trans. Circ. Syst. I: Reg. Papers 59, 1 (2012), 148--158.Google ScholarCross Ref
- Shahar Kvatinsky, Eby G. Friedman, Avinoam Kolodny, and Uri C. Weiser. 2013. TEAM: Threshold adaptive memristor model. IEEE Trans. Circ. Syst. I: Reg. Papers 60, 1 (2013), 211--221.Google ScholarCross Ref
- Yann LeCun. 1998. The MNIST database of handwritten digits. Retrieved from http://yann.lecun.com/exdb/mnist/.Google Scholar
- M. Lichman. 2013. UCI Machine Learning Repository. Retrieved from http://archive.ics.uci.edu/ml.Google Scholar
- Roberto Marani, Gennaro Gelao, and Anna Gina Perri. 2015. A review on memristor applications. arXiv Preprint arXiv:1506.06899 (2015).Google Scholar
- Cory Merkel. 2017. Current-mode memristor crossbars for neuromemristive systems. arXiv Preprint arXiv:1707.05316 (2017).Google Scholar
- Cory Merkel and Dhireesha Kudithipudi. 2014. Neuromemristive extreme learning machines for pattern classification. In Proceedings of the IEEE Computer Society Annual Symposium on VLSI (ISVLSI’14). IEEE, 77--82. Google ScholarDigital Library
- Yoh-Han Pao, Gwang-Hoon Park, and Dejan J. Sobajic. 1994. Learning and generalization characteristics of the random vector functional-link net. Neurocomputing 6, 2 (1994), 163--180.Google ScholarCross Ref
- André Bannwart Perina, Paulo Matias, Eduardo Marques, Vanderlei Bonato, João Miguel Gago Pontes De Brito, et al. 2017. Exploiting kant and kimura matrix inversion algorithm on FPGA. In Proceedings of the Euromicro Conference on Digital System Design (DSD’17). IEEE, 516--519.Google Scholar
- Mirko Prezioso, Farnood Merrikh-Bayat, B. D. Hoskins, G. C. Adam, Konstantin K. Likharev, and Dmitri B. Strukov. 2015. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 7550 (2015), 61--64.Google Scholar
- Maheshwar Pd. Sah, Changju Yang, Hyongsuk Kim, and Leon O. Chua. 2012. Memristor circuit for artificial synaptic weighting of pulse inputs. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS’12). IEEE, 1604--1607.Google Scholar
- Greg S. Snider. 2008. Spike-timing-dependent learning in memristive nanodevices. In Proceedings of the IEEE International Symposium on Nanoscale Architectures (NANOARCH’08). IEEE, 85--92. Google ScholarDigital Library
- Daniel Soudry, Dotan Di Castro, Asaf Gal, Avinoam Kolodny, and Shahar Kvatinsky. 2015. Memristor-based multilayer neural networks with online gradient descent training. IEEE Trans. Neural Netw. Learn. Syst. 26, 10 (2015), 2408--2421.Google ScholarCross Ref
- Dmitri B. Strukov, Gregory S. Snider, Duncan R. Stewart, and R. Stanley Williams. 2008. The missing memristor found. Nature 453, 7191 (2008), 80--83.Google Scholar
- Manan Suri, Vivek Parmar, Gilbert Sassine, and Fabien Alibart. 2015. OXRAM based ELM architecture for multi-class classification applications. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’15). IEEE, 1--8.Google ScholarCross Ref
- Tarek M. Taha, Raqibul Hasan, and Chris Yakopcic. 2014. Memristor crossbar-based multicore neuromorphic processors. In Proceedings of the 27th IEEE International System-on-Chip Conference (SOCC’14). IEEE, 383--389.Google ScholarCross Ref
- Enyi Yao and Arindam Basu. 2017. VLSI extreme learning machine: A design space exploration. IEEE Trans. Very Large Scale Integr. Syst. 25, 1 (2017), 60--74. Google ScholarDigital Library
- Abdullah M. Zyarah and Dhireesha Kudithipudi. 2017. Extreme learning machine as a generalizable classification engine. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’17). IEEE, 3371--3376.Google Scholar
- Abdullah M. Zyarah, Nicholas Soures, Lydia Hays, Robin B. Jacobs-Gedrim, Sapan Agarwal, Matthew Marinella, and Dhireesha Kudithipudi. 2017. Ziksa: On-chip learning accelerator with memristor crossbars for multilevel neural networks. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS’17). IEEE, 1--4.Google ScholarCross Ref
Index Terms
- Semi-Trained Memristive Crossbar Computing Engine with In Situ Learning Accelerator
Recommendations
Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars
ISLPED '22: Proceedings of the ACM/IEEE International Symposium on Low Power Electronics and DesignSpiking Neural Networks (SNNs) have recently emerged as the low-power alternative to Artificial Neural Networks (ANNs) owing to their asynchronous, sparse, and binary information processing. To improve the energy-efficiency and throughput, SNNs can be ...
Memristive Crossbar Mapping for Neuromorphic Computing Systems on 3D IC
In recent years, neuromorphic computing systems based on memristive crossbar have provided a promising solution to enable acceleration of neural networks. However, most of the neural networks used in realistic applications are often sparse. If such ...
Comparison of two memristor based neural network learning schemes for crossbar architecture
IWANN'13: Proceedings of the 12th international conference on Artificial Neural Networks: advances in computational intelligence - Volume Part IThis paper compares two neural network learning schemes in crossbar architecture, using memristive elements. Novel memristive crossbar architecture with dense synaptic connections suitable for online training was developed. Training algorithms and ...
Comments