Amir Morad
Skip Abstract Section
Abstract
GP-SIMD, a novel hybrid general-purpose SIMD computer architecture, resolves the issue of data synchronization by in-memory computing through combining data storage and massively parallel processing. GP-SIMD employs a two-dimensional access memory with modified SRAM storage cells and a bit-serial processing unit per each memory row. An analytic performance model of the GP-SIMD architecture is presented, comparing it to associative processor and to conventional SIMD architectures. Cycle-accurate simulation of four workloads supports the analytical comparison. Assuming a moderate die area, GP-SIMD architecture outperforms both the associative processor and conventional SIMD coprocessor architectures by almost an order of magnitude while consuming less power.
- A. Akerib and R. Adar. 1995. Associative approach to real time color, motion and stereo vision. In Proceedings of the 1995 International Conference on Acoustics, Speech, and Signal Processing (ICASSP’95). Vol. 5. IEEE.Google Scholar
- A. J. Akerib and S. Ruhman. 1991. Associative array and tree algorithms in stereo vision. In Proceedings of the 8th Israel Conference on Artificial Intelligence, Vision & Pattern Recognition. Elsevier.Google Scholar
- G. Almási et al. 2003. Dissecting Cyclops: A detailed analysis of a multithreaded architecture. ACM SIGARCH Computer Architecture News 31, 1, 26--38. Google ScholarDigital Library
- AltiVec Engine. 2014. Homepage. Retrieved from http://www.freescale.com/webapp/sps/site/overview.jsp?code=DRPPCALTVC.Google Scholar
- ARM. 2014. NEON™ General-Purpose SIMD Engine. Retrieved from http://www.arm.com/products/processors/technologies/neon.php.Google Scholar
- C. Auth et al. 2012. A 22nm high performance and low-power CMOS technology featuring fully-depleted tri-gate transistors, self-aligned contacts and high density MIM capacitors. In Proceedings of the 2012 Symposium on VLSI Technology (VLSIT’12). IEEE.Google ScholarCross Ref
- K. Banerjee et al. 2003. A self-consistent junction temperature estimation methodology for nanometer scale ICs with implications for performance and thermal management. Electron Devices Meeting, 2003. IEDM'03 Technical Digest. IEEE International. IEEE.Google ScholarCross Ref
- K. E. Batcher. 1974. STARAN parallel processor system hardware. In Proceedings of the National Computer Conference. 405--410. Google ScholarDigital Library
- N. Binkert et al. 2011. The gem5 simulator. ACM SIGARCH Computer Architecture News 39, 2, 1--7. Google ScholarDigital Library
- F. Black and M. Scholes. 1973. The pricing of options and corporate liabilities. Journal of Political Economy 81, 637--654.Google ScholarCross Ref
- S. Borkar. 2007. Thousand core chips: A technology perspective. In Proceedings of the ACM/IEEE 44th Design Automation Conference (DAC’07). 746--749. Google ScholarDigital Library
- J. Brockman et al. 2004. A low cost, multithreaded processing-in-memory system. In Proceedings of the 31st International Symposium on Computer Architecture. Google ScholarDigital Library
- D. T. Burger Austin. 1997. The SimpleScalar tool set, version 2.0. ACM SIGARCH Computer Architecture News 25, 3, 13--25. Google ScholarDigital Library
- A. Cassidy and A. Andreou. 2012. Beyond Amdahl Law - An objective function that links performance gains to delay and energy. IEEE Transactions on Computers 61, 8, 1110--1126. Google ScholarDigital Library
- E. L. Cloud. 1988. The geometric arithmetic parallel processor. In Proceedings of the 2nd Symposium on the Frontiers of Massively Parallel Computation. IEEE.Google Scholar
- P. Dlugosch, D. Brown, P. Glendenning, M. Leventhal, and H. Noyes. 2014. An efficient and scalable semiconductor architecture for parallel automata processing. In IEEE Transactions on Parallel and Distributed Systems. 1--1.Google Scholar
- J. Draper et al. 2002. The architecture of the DIVA processing-in-memory chip. In Proceedings of the 16th International Conference on Supercomputing. ACM. Google ScholarDigital Library
- H. Esmaeilzadeh et al. 2013. Power challenges may end the multicore era. Communications of the ACM 56, 2, 93--102. Google ScholarDigital Library
- H. Flatt et al. 1989. Performance of parallel processors. Parallel Computing 12, 1, 1--20.Google ScholarCross Ref
- C. Foster. 1976. Content Addressable Parallel Processors. Van Nostrand Reinhold Company, New York. Google ScholarDigital Library
- M. Gokhale et al. 1995. Processing in memory: The Terasys massively parallel PIM array. IEEE Computer 23--31. Google ScholarDigital Library
- M. Gschwind et al. 2006. Synergistic processing in cell's multicore architecture. IEEE Micro 26, 2, 10--24. Google ScholarDigital Library
- N. Gunther, S. Subramanyam, and S. Parvu. 2011. A methodology for optimizing multithreaded system scalability on multi--cores. Retrieved from http://arxiv.org/abs/1105.4301.Google Scholar
- M. Hall et al. 1999. Mapping irregular applications to DIVA, a PIM-based data-intensive architecture. In Proceedings of the ACM/IEEE Conference on Supercomputing. Google ScholarDigital Library
- N. Hardavellas et al. 2011. Toward dark silicon in servers. IEEE Micro 31, 4, 6--15. Google ScholarDigital Library
- J. Hennessy and D. A. Patterson. 1996. Computer Architecture: A Quantitative Approach (2nd ed.) Morgan Kaufmann Publishers. Google ScholarDigital Library
- D. Hentrich et al. 2009. Performance evaluation of SRAM cells in 22nm predictive CMOS technology. In Proceedings of the IEEE International Conference on Electro/Information Technology.Google ScholarCross Ref
- M. Hill et al. 2008. Amdahl's law in the multicore era. IEEE Computer 41, 7, 33--38. Google ScholarDigital Library
- S. Hong and H. Kim. 2009. An analytical model for a GPU architecture with memory-level and thread-level parallelism awareness. ACM SIGARCH Computer Architecture News 37, 3. Google ScholarDigital Library
- IBM. 2005. PowerPC Vector/SIMD Multimedia Extension. Retrieved from http://math-at-las.sourceforge.net/devel/assembly/vector_simd_pem.ppc.2005AUG23.pdf.Google Scholar
- Intel. 2013. The Intel® Xeon Phi™ Coprocessor. Retrieved from http://www.intel.com/content/www/us/en/high-performance-computing/high-performance-xeon-phi-coprocessor-brief.html.Google Scholar
- S. W. Keckler et al. 2011. GPUs and the future of parallel computing. IEEE Micro 31, 5, 7--17. Google ScholarDigital Library
- P. Kogge et al. 2000. PIM architectures to support petaflops level computation in the HTMT machine. In Proceedings of the International Workshop on Innovative Architecture for Future Generation Processors and Systems. Google ScholarDigital Library
- C. E. Kozyrakis et al. 1997. Scalable processors in the billion-transistor era: IRAM. Computer 30, 9, 75--78. Google ScholarDigital Library
- S. Kumar. 2012. Smart Memory. Retrieved from http://www.hotchips.org/wp-content/uploads/hc_archives/hc22/HC22.23.325-1-Kumar-Smart-Memory.pdf.Google Scholar
- G. Lipovski and C. Yu. 1999. The dynamic associative access memory chip and its application to SIMD processing and full-text database retrieval. In Proceedings of the IEEE International Workshop on Memory Technology, Design and Testing. Google ScholarDigital Library
- G. Loh. 2008. The cost of uncore in throughput-oriented many-core processors. In Proceedings of the Workshop on Architectures and Languages for Throughput Applications (ALTA).Google Scholar
- D. Luebke. 2004. General-purpose computation on graphics hardware. In Proceedings of the SIGGRAPH Workshop. Google ScholarDigital Library
- T. Midwinter, M. Huch, P. A. Ivey, and G. Saucier. 1988. Architectural considerations of a wafer scale processor. IEE Colloquium on VLSI for Parallel Processing 4/1, 4/4, 17.Google Scholar
- A. Morad et al. 2013. Generalized multiAmdahl: Optimization of heterogeneous multi-accelerator SoC. Computer Architecture Letters 13, 1, 37--40.Google ScholarDigital Library
- A. Morad et al. 2014. Convex optimization of resource allocation in asymmetric and heterogeneous SoC. Power and Timing Modeling, Optimization and Simulation (PATMOS).Google Scholar
- A. Morad et al. 2014. Efficient dense and sparse matrix multiplication on GP-SIMD. Power and Timing Modeling, Optimization and Simulation (PATMOS).Google Scholar
- A. Morad et al. 2014. Optimization of asymmetric and heterogeneous SoC. Under review.Google Scholar
- T. Morad et al. 2006. Performance, power efficiency and scalability of asymmetric cluster chip multiprocessors. IEEE Computer Architecture Letters 5, 1, 14--17. Google ScholarDigital Library
- J. Owens et al. 2008. GPU computing. Proceedings of the IEEE 96, 5, 879--899.Google ScholarCross Ref
- A. Pedram. 2013. Algorithm/Architecture Codesign of Low Power and High Performance Linear Algebra Compute Fabrics. PhD dissertation, University of Texas. Retrieved from http://repositories.lib.utexas.edu/bitstream/handle/2152/21364/PEDRAM-DISSERTATION-2013.pdf?sequence=1.Google Scholar
- F. Pollack. 1999. New microarchitecture challenges in the coming generations of CMOS process technologies. Proceedings of the 32nd Annual ACM/IEEE International Symposium on Microarchitecture. IEEE Computer Society. Google ScholarDigital Library
- J. Potter et al. 1994. ASC: An associative-computing paradigm. Computer 27, 11, 19--25. Google ScholarDigital Library
- S. Pugsley et al. 2014. Comparing implementations of near-data computing with in-memory MapReduce workloads. IEEE Micro 34, 4, 44--52.Google ScholarCross Ref
- G. Qing, X. Guo, R. Patel, E. Ipek, and E. Friedman. 2013. AP-DIMM: Associative computing with STT-MRAM. In Proceedings of the Annual International Symposium on Computer Architecture (ISCA’13). ACM, New York, NY. Google ScholarDigital Library
- M. Quinn. 1987. Designing Efficient Algorithms for Parallel Computers. McGraw-Hill, 125. Google ScholarDigital Library
- S. F. Reddaway. 1973. DAP—a distributed array processor. ACM SIGARCH Computer Architecture News 2, 4, 61--65. Google ScholarDigital Library
- B. Rogers et al. 2009. Scaling the bandwidth wall: Challenges in and avenues for CMP scaling. In Proceedings of the 36th Annual International Symposium on Computer Architecture (ISCA’09). ACM, New York, NY, 371--382. Google ScholarDigital Library
- R. M. Russell. 1978. The CRAY-1 computer system. Communications of the ACM 21, 1, 63--72. Google ScholarDigital Library
- G. E. Sayre. 1976. STARAN: An associative approach to multiprocessor architecture. Computer Architecture. Springer, Berlin.Google Scholar
- I. Scherson et al. 1992. Bit-parallel arithmetic in a massively-parallel associative processor. IEEE Transactions on Computers 41, 10. Google ScholarDigital Library
- J. Sheaffer et al. 2005. Studying thermal management for graphics-processor architectures. ISPASS. Google ScholarDigital Library
- D. Steinkraus, L. Buck, and P. Simard. 2005. Using GPUs for machine learning algorithms. IEEE ICDAR. Google ScholarDigital Library
- T. Sterling and H. Zima. 2002. Gilgamesh: A multithreaded processor-in-memory architecture for petaflops computing. In Proceedings of the ACM/IEEE Conference on Supercomputing. Google ScholarDigital Library
- J. Suh et al. 2001. A PIM-based multiprocessor system. In Proceedings of the 15th International Symposium on Parallel and Distributed Processing. Google ScholarDigital Library
- L. W. Tucker and G. G. Robertson. 1988. Architecture and applications of the connection machine. Computer 21, 8, 26--38. Google ScholarDigital Library
- V. Volkov and J. W. Demmel. 2008. Benchmarking GPUs to tune dense linear algebra. In Proceedings of the 2008 ACM/IEEE Conference on Supercomputing. IEEE Press. Google ScholarDigital Library
- D. Wentzlaff et al. 2010. Core Count vs. Cache Size for Manycore Architectures in the Cloud. Technical Report. MIT-CSAIL-TR-2010-008, MIT.Google Scholar
- L. Yavits. 1994. Architecture and Design of Associative Processor for Image Processing and Computer Vision. MSc Thesis, Technion -- Israel Institute of Technology. Retrieved from http://webee.technion.ac.il/∼ran/papers/LeonidYavitsMasterThesis1994.pdf.Google Scholar
- L. Yavits et al. 2014a. Computer architecture with associative processor replacing last level cache and SIMD accelerator. IEEE Transactions on Computers.Google Scholar
- L. Yavits et al. 2014b. The effect of communication and synchronization on Amdahl's law in multicore systems. Parallel Computing 40.1, 1--16. Google ScholarDigital Library
- L. Yavits et al. 2014c. Thermal analysis of 3D associative processor. http://arxiv.org/abs/1307.3853v1Google Scholar
- D. Zhang et al. 2014. TOP-PIM: throughput-oriented programmable processing in memory. In Proceedings of the 23rd International Symposium on High-Performance Parallel and Distributed Computing. ACM. Google ScholarDigital Library
- Y. Zhang and J. D. Owens. 2011. A quantitative performance analysis model for GPU architectures. In Proceedings of the 2011 IEEE 17th International Symposium on High Performance Computer Architecture (HPCA). IEEE. Google ScholarDigital Library
Index Terms
- GP-SIMD Processing-in-Memory
Recommendations
Resistive GP-SIMD Processing-In-Memory
GP-SIMD, a novel hybrid general-purpose SIMD architecture, addresses the challenge of data synchronization by in-memory computing, through combining data storage and massive parallel processing. In this article, we explore a resistive implementation of ...
Exploring Processing In-Memory for Different Technologies
GLSVLSI '19: Proceedings of the 2019 on Great Lakes Symposium on VLSIThe recent emergence of IoT has led to a substantial increase in the amount of data processed. Today, a large number of applications are data intensive, involving massive data transfers between processing core and memory. These transfers act as a ...
Exploring Time and Energy for Complex Accesses to a Hybrid Memory Cube
MEMSYS '16: Proceedings of the Second International Symposium on Memory SystemsThrough-Silicon Vias (TSVs) and three-dimensional die stacking technologies are enabling a combination of DRAM and CMOS die layer within a single stack, leading to stacked memory. Functionality that was previously associated with the microprocessor, ...
Comments