ABSTRACT
Mantle convection is the fundamental physical process within earth's interior responsible for the thermal and geological evolution of the planet, including plate tectonics. The mantle is modeled as a viscous, incompressible, non-Newtonian fluid. The wide range of spatial scales, extreme variability and anisotropy in material properties, and severely nonlinear rheology have made global mantle convection modeling with realistic parameters prohibitive. Here we present a new implicit solver that exhibits optimal algorithmic performance and is capable of extreme scaling for hard PDE problems, such as mantle convection. To maximize accuracy and minimize runtime, the solver incorporates a number of advances, including aggressive multi-octree adaptivity, mixed continuous-discontinuous discretization, arbitrarily-high-order accuracy, hybrid spectral/geometric/algebraic multigrid, and novel Schur-complement preconditioning. These features present enormous challenges for extreme scalability. We demonstrate that---contrary to conventional wisdom---algorithmically optimal implicit solvers can be designed that scale out to 1.5 million cores for severely nonlinear, ill-conditioned, heterogeneous, and anisotropic PDEs.
- D. Depaolo, T. Cerling, S. Hemming, A. Knoll, F. Richter, L. Royden, R. Rudnick, L. Stixrude, and J. Trefil, "Origin and Evolution of Earth: Research Questions for a Changing Planet," National Academies Press, Committee on Grand Research Questions in the Solid Earth Sciences, National Research Council of the National Academies, 2008.Google Scholar
- C. Burstedde, O. Ghattas, M. Gurnis, T. Isaac, G. Stadler, T. Warburton, and L. C. Wilcox, "Extreme-scale AMR," in Proceedings of SC10. ACM/IEEE, 2010. Google ScholarDigital Library
- L. Alisic, M. Gurnis, G. Stadler, C. Burstedde, and O. Ghattas, "Multi-scale dynamics and rheology of mantle flow with plates," Journal of Geophysical Research, vol. 117, p. B10402, 2012.Google ScholarCross Ref
- G. Stadler, M. Gurnis, C. Burstedde, L. C. Wilcox, L. Alisic, and O. Ghattas, "The dynamics of plate tectonics and mantle flow: From local to global scales," Science, vol. 329, no. 5995, pp. 1033--1038, 2010.Google ScholarCross Ref
- L. Alisic, M. Gurnis, G. Stadler, C. Burstedde, L. C. Wilcox, and O. Ghattas, "Slab stress and strain rate as constraints on global mantle flow," Geophysical Research Letters, vol. 37, p. L22308, 2010.Google ScholarCross Ref
- C. Burstedde, L. C. Wilcox, and O. Ghattas, "p4est: Scalable algorithms for parallel adaptive mesh refinement on forests of octrees," SIAM Journal on Scientific Computing, vol. 33, no. 3, pp. 1103--1133, 2011. Google ScholarDigital Library
- J. Dongarra, J. Hittinger, J. Bell, L. Chacón, R. Falgout, M. Heroux, P. Hovland, E. Ng, C. Webster, and S. Wild, "Applied mathematics research for exascale computing," Report of the DOE/ASCR Exascale Mathematics Working Group, 2014.Google ScholarCross Ref
- S. C. Eisenstat and H. F. Walker, "Choosing the forcing terms in an inexact Newton method," SIAM Journal on Scientific Computing, vol. 17, pp. 16--32, 1996. Google ScholarDigital Library
- B. Gmeiner, U. Rüde, H. Stengel, C. Waluga, and B. Wohlmuth, "Performance and scalability of Hierarchical Hybrid Multigrid solvers for Stokes systems," SIAM Journal on Scientific Computing, vol. 37, no. 2, pp. C143--C168, 2015.Google ScholarCross Ref
- S. Reiter, A. Vogel, I. Heppner, M. Rupp, and G. Wittum, "A massively parallel geometric multigrid solver on hierarchically distributed grids," Computing and Visualization in Science, vol. 16, no. 4, pp. 151--164, 2013. Google ScholarDigital Library
- A. H. Baker, R. D. Falgout, T. V. Kolev, and U. M. Yang, "Scaling hypre's multigrid solvers to 100,000 cores," in High-Performance Scientific Computing, M. W. Berry, K. A. Gallivan, E. Gallopoulos, A. Grama, B. Philippe, Y. Saad, and F. Saied, Eds. Springer London, 2012, pp. 261--279.Google Scholar
- S. Badia, A. F. Martín, and J. Principe, "A highly scalable parallel implementation of balancing domain decomposition by constraints," SIAM Journal on Scientific Computing, vol. 36, no. 2, pp. C190--C218, 2014.Google ScholarCross Ref
- O. Ippisch and M. Blatt, "Scalability test of μϕ and the Parallel Algebraic Multigrid solver of DUNE-ISTL," in Jülich Blue Gene/P Extreme Scaling Workshop, no. FZJ-JSC-IB-2011-02. Jülich Supercomputing Centre, 2011.Google Scholar
- H. Sundar, G. Biros, C. Burstedde, J. Rudi, O. Ghattas, and G. Stadler, "Parallel geometric-algebraic multigrid on unstructured forests of octrees," in Proceedings of SC12. ACM/IEEE, 2012. Google ScholarDigital Library
- C. Burstedde, O. Ghattas, M. Gurnis, E. Tan, T. Tu, G. Stadler, L. C. Wilcox, and S. Zhong, "Scalable adaptive mantle convection simulation on petascale supercomputers," in Proceedings of SC08. ACM/IEEE, 2008. Google ScholarDigital Library
- M. Kronbichler, T. Heister, and W. Bangerth, "High accuracy mantle convection simulation through modern numerical methods," Geophysical Journal International, vol. 191, no. 1, pp. 12--29, 2012.Google ScholarCross Ref
- G. Ranalli, Rheology of the Earth. Springer, 1995.Google Scholar
- T. Isaac, C. Burstedde, L. C. Wilcox, and O. Ghattas, "Recursive algorithms for distributed forests of octrees," SIAM Journal on Scientific Computing (to appear), 2015, http://arxiv.org/abs/1406.0089.Google Scholar
- M. O. Deville, P. F. Fischer, and E. H. Mund, High-Order Methods for Incompressible Fluid Flow, ser. Cambridge Monographs on Applied and Computational Mathematics. Cambridge, UK: Cambridge University Press, 2002, vol. 9.Google ScholarCross Ref
- M. Benzi, G. H. Golub, and J. Liesen, "Numerical solution of saddle point problems," Acta Numerica, vol. 14, pp. 1--137, 2005.Google ScholarCross Ref
- H. Elman, V. Howle, J. Shadid, R. Shuttleworth, and R. Tuminaro, "Block preconditioners based on approximate commutators," SIAM Journal on Scientific Computing, vol. 27, no. 5, pp. 1651--1668, 2006. Google ScholarDigital Library
- D. A. May and L. Moresi, "Preconditioned iterative methods for Stokes flow problems arising in computational geodynamics," Physics of the Earth and Planetary Interiors, vol. 171, pp. 33--47, 2008.Google ScholarCross Ref
- D. Rossinelli, B. Hejazialhosseini, P. Hadjidoukas, C. Bekas, A. Curioni, A. Bertsch, S. Futral, S. J. Schmidt, N. A. Adams, and P. Koumoutsakos, "11 pflop/s simulations of cloud cavitation collapse," in Proceedings of SC13. ACM/IEEE, 2013. Google ScholarDigital Library
- V. Karakasis, T. Gkountouvas, K. Kourtis, G. I. Goumas, and N. Koziris, "An extended compression format for the optimization of sparse matrix-vector multiplication," IEEE Trans. Parallel Distrib. Syst., vol. 24, no. 10, pp. 1930--1940, 2013. Google ScholarDigital Library
- J. Milano and P. Lembke, "IBM system Blue Gene solution: Blue Gene/Q hardware overview and installation planning," IBM, Tech. Rep. SG24-7872-01, May 2013.Google Scholar
- K. Yoshii, K. Iskra, R. Gupta, P. Beckman, V. Vishwanath, C. Yu, and S. Coghlan, "Evaluating power-monitoring capabilities on IBM Blue Gene/P and Blue Gene/Q," in Proc. of the IEEE Int. Conf. Cluster Computing, Beijing, 2012, pp. 36--44. Google ScholarDigital Library
- U.S. Energy Information Administration (EIA), "Electric power monthly, with data from january 2015," U.S. Department of Energy, Tech. Rep., March 2015. {Online}. Available: http://www.eia.gov/electricity/monthly/pdf/epm.pdfGoogle Scholar
- S. Zhong and M. Gurnis, "Controls on trench topography from dynamic models of subducted slabs," J. Geophys. Res., vol. 99, pp. 15 683--15 695, 1994.Google ScholarCross Ref
- T. R. A. Song and M. Simons, "Large trench-parallel gravity variations predict seismogenic behavior in subduction zones," Science, vol. 301, pp. 630--633, 2003.Google ScholarCross Ref
- V. Ratnaswamy, G. Stadler, and M. Gurnis, "Adjoint-based estimation of plate coupling in a non-linear mantle flow model: theory and examples," Geophysical Journal International, vol. 202, no. 2, pp. 768--786, 2015.Google ScholarCross Ref
- L. Ruff and H. Kanamori, "Seismic coupling and uncoupling at subduction zones," Tectonophysics, vol. 99, no. 2, pp. 99--117, 1983.Google ScholarCross Ref
- C. H. Scholz and J. Campos, "The seismic coupling of subduction zones revisited," Journal of Geophysical Research: Solid Earth, vol. 117, no. B5, 2012.Google Scholar
- D. Forsyth and S. Uyeda, "On the relative importance of the driving forces of plate motion," Geophysical Journal International, vol. 43, no. 1, pp. 163--200, 1975.Google ScholarCross Ref
- B. Hager and R. O'Connell, "A simple global model of plate dynamics and mantle convection," J. Geophys. Res., vol. 86, pp. 4843--4867, 1981.Google ScholarCross Ref
Index Terms
- An extreme-scale implicit solver for complex PDEs: highly heterogeneous flow in earth's mantle
Recommendations
An interior penalty stabilised incompressible discontinuous GalerkinFourier solver for implicit large eddy simulations
We present an implicit Large Eddy Simulation (iLES) h / p high order (2) unstructured Discontinuous GalerkinFourier solver with sliding meshes. The solver extends the laminar version of Ferrer and Willden, 2012 34, to enable the simulation of turbulent ...
Implementation of density-based implicit LU-SGS solver in the framework of OpenFOAM
Implicit steady and dual-time LU-SGS solver is implemented in OpenFOAM.The implementation codes of the sweep process in LU-SGS approach are introduced.Typical test cases are used to investigate the performance of LU-SGS solver.Convergence performance of ...
Comments