Abstract
Supercomputing is evolving toward hybrid and accelerator-based architectures with millions of cores. The Hardware/Hybrid Accelerated Cosmology Code (HACC) framework exploits this diverse landscape at the largest scales of problem size, obtaining high scalability and sustained performance. Developed to satisfy the science requirements of cosmological surveys, HACC melds particle and grid methods using a novel algorithmic structure that flexibly maps across architectures, including CPU/GPU, multi/many-core, and Blue Gene systems. In this Research Highlight, we demonstrate the success of HACC on two very different machines, the CPU/GPU system Titan and the BG/Q systems Sequoia and Mira, attaining very high levels of scalable performance. We demonstrate strong and weak scaling on Titan, obtaining up to 99.2% parallel efficiency, evolving 1.1 trillion particles. On Sequoia, we reach 13.94 PFlops (69.2% of peak) and 90% parallel efficiency on 1,572,864 cores, with 3.6 trillion particles, the largest cosmological benchmark yet performed. HACC design concepts are applicable to several other supercomputer applications.
- Bhattacharya, S., Habib, S., Heitmann, K., Vikhlinin, A. Dark matter Halo profiles of massive clusters: Theory versus observations. Astrophys. J. 766 (2013), 32.Google ScholarCross Ref
- Bryan, G.L., Norman, M.L. In 12th Kingston Meeting on Theoretical Astrophysics, Proceedings of Meeting Held in Halifax; Nova Scotia (ASP Conference Series # 123), D.A. Clarke and M. Fall, eds. 1996; see also O'Shea, B.W., Nagamine, K., Springel, V., Hernquist, L., Norman, M.L. Astrophys. J. Supp. 160 (2005), 1.Google Scholar
- Couchman, H.M.P., Thomas, P.A., Pearce, F.R. Hydra: An adaptive-mesh implementation of P 3M-SPH Astrophys. J. 452, 797 (1995).Google Scholar
- For a review of cosmological simulation methods, see also Dolag, K., Borgani, S., Schindler, S., Diaferio, A., Bykov, A.M. Space Sci. Rev. 134 (2008), 229.Google Scholar
- Fryxell, B., et al. FLASH: An adaptive mesh hydrodynamics code for modeling astrophysical thermonuclear flashes. Astrophys. J. Supp. 131 (2000), 273.Google ScholarCross Ref
- Gafton, E., Rosswog, S. A fast recursive coordinate bisection tree for neighbour search and gravity. Mon. Not. R. Astron. Soc. 418 (2011), 770.Google ScholarCross Ref
- Habib, S., Morozov, V., Finkel, H., Pope, A., Heitmann, K., Kumaran, K., Peterka, T., Insley, J., Daniel, D., Fasel, P., Frontiere, N., Lukić, Z. arXiv:1211.4864, Supercomputing 2012.Google Scholar
- Habib, S., Pope, A., Finkel, H., Frontiere, N., Heitmann, K., Daniel, D., Fasel, P., Morozov, V., Zagaris, G., Peterka, T., Vishwanath, V., Lukić, Z., Sehrish, S., Liao, W.-K. HACC: Simulating sky surveys on state-of-the-art supercomputing architectures. New Astron. 42 (2016), 49 arXiv:1410.2805 {astro-ph.IM}.Google ScholarCross Ref
- Habib, S., Pope, A., Lukić, Z., Daniel, D., Fasel, P., Desai, N., Heitmann, K., Hsu, C.-H., Ankeny, L., Mark, G., Bhattacharya, S., Ahrens, J. Hybrid petacomputing meets cosmology: The Roadrunner Universe project. J. Phys. Conf. Ser. 180 (2009), 012019.Google ScholarCross Ref
- Hamming, R.W. Digital Filters. Dover, Publications, Mineola, New York 1998.Google Scholar
- Heitmann, K., Frontiere, N., Sewell, C., Habib, S., Pope, A., Finkel, H., Rizzi, S., Insley, J., Bhattacharya, S. The Q continuum simulation: Harnessing the power of GPU accelerated supercomputers. J. - Astrophys. J. Supp. 219 (2015), 34 arXiv:1411.3396 {astro-ph.CO}.Google Scholar
- Heitmann, K., Higdon, D., White, M., Habib, S., Williams, B.J., Lawrence, E., Wagner, C. The Coyote universe. II. Cosmological models and precision emulation of the nonlinear matter power spectrum Astrophys. J. 705 (2009), 156.Google Scholar
- Heitmann, K., Lukić, Z., Fasel, P., Habib, S., Warren, M.S., White, M., Ahrens, J., Ankeny, L., Armstrong, R., O'Shea, B., Ricker, P.M., Springel, V., Stadel, J., Trac, H. The cosmic code comparison project. Comput. Sci. Dis. 1 (2008), 015003.Google ScholarCross Ref
- Heitmann, K., Ricker, P.M., Warren, M.S., Habib, S. Robustness of cosmological simulations. I. Large-scale Structure. Astrophys. J. Supp. 160 (2005), 28.Google ScholarCross Ref
- Hockney, R.W., Eastwood, J.W. Computer Simulation Using Particles. Adam Hilger, New York, 1988. Google ScholarDigital Library
- Peebles, P.J.E., Structure of the coma cluster of galaxies. Astron. J. 75 (1970), 13.Google ScholarCross Ref
- Peebles, P.J.E. The Large-Scale Structure of the Universe. Princeton University Press, Princeton, New Jersey 1980.Google Scholar
- Pfalzner, S., Gibbon, P. Many-Body Tree Methods in Physics. Cambridge University Press, 1996; see also Barnes, J., Hut, P. Nature 324, 446 (1986); Warren, M.S., Salmon, J.K. Technical Paper, Supercomputing, Cambridge University Press, New York, USA 1993.Google Scholar
- Pope, A., Habib, S., Lukic, Z., Daniel, D., Fasel, P., Desai, N., Heitmann, K. Comput. Sci. Eng. 12 (2010), 17.Google ScholarDigital Library
- The accelerated universe. Power, C., Navarro, J.F., Jenkins, A., Frenk, C.S., White, S.D.M., Springel, V., Stadel, J., Quinn, T. The inner structure of ACDM haloes - I. A numerical convergence study. Mon. Not. R. Astron. Soc. 338 (2003), 14.Google Scholar
- Shandarin, S.F., Zeldovich, Ya.B. The large-scale structure of the universe: Turbulence, intermittency, structures in a self-gravitating medium. Rev. Mod. Phys. 61 (1989), 185.Google ScholarCross Ref
- Springel, V. The cosmological simulation code GADGET-2. Mon. Not. R. Astron. Soc. 364 (2005), 1105.Google ScholarCross Ref
- Teyssier, R. Cosmological hydrodynamics with adaptive mesh refinement. A new high resolution code called RAMSES. A&A 385 (2002), 337.Google ScholarCross Ref
- White, M. The mass of a halo. Astron. and Astrophys. 367 (2001), 27.Google ScholarCross Ref
- White, M., Pope, A., Carlson, J., Heitmann, K., Habib, S., Fasel, P., Daniel, D., Lukić, Z. Particle mesh simulations of the Lyα forest and the signature of Baryon acoustic oscillations in the intergalactic medium. Astrophys. J. 713 (2010), 383.Google ScholarCross Ref
Index Terms
- HACC: extreme scaling and performance across diverse architectures
Recommendations
HACC: extreme scaling and performance across diverse architectures
SC '13: Proceedings of the International Conference on High Performance Computing, Networking, Storage and AnalysisSupercomputing is evolving towards hybrid and accelerator-based architectures with millions of cores. The HACC (Hardware/Hybrid Accelerated Cosmology Code) framework exploits this diverse landscape at the largest scales of problem size, obtaining high ...
A Performance-Portable SYCL Implementation of CRK-HACC for Exascale
SC-W '23: Proceedings of the SC '23 Workshops of The International Conference on High Performance Computing, Network, Storage, and AnalysisThe first generation of exascale systems will include a variety of machine architectures, featuring GPUs from multiple vendors. As a result, many developers are interested in adopting portable programming models to avoid maintaining multiple versions of ...
Evaluation of Rodinia Codes on Intel Xeon Phi
ISMS '13: Proceedings of the 2013 4th International Conference on Intelligent Systems, Modelling and SimulationHigh performance computing (HPC) is a niche area where various parallel benchmarks are constantly used to explore and evaluate the performance of Heterogeneous computing systems on the horizon. The Rodinia benchmark suite, a collection of parallel ...
Comments