The impact of work distribution on in situ visualization: a case study

ABSTRACT

Large-scale computer simulations generate data at rates that necessitate visual analysis tools to run in situ. The distribution of work on and across nodes of a supercomputer is crucial to utilize compute resources as efficiently as possible. In this paper, we study two work distribution problems in the context of in situ visualization and jointly assess the performance impact of different variants. First, especially for simulations involving heterogeneous loads across their domain, dynamic load balancing can significantly reduce simulation run times. However, the adjustment of the domain partitioning associated with this also has a direct impact on visualization performance. The exact impact of this side effect is largely unclear a priori as generally different criteria are used for balancing simulation and visualization load. Second, on node level, the adequate allocation of threads to simulation or visualization tasks minimizes the performance drain of the simulation while also enabling timely visualization results. In our case study, we jointly study both work distribution aspects with the visualization framework MegaMol coupled in situ on node level to the molecular dynamics simulation ls1 Mardyn on Stampede2 at TACC.

References

1. Hasan Abbasi, Matthew Wolf, Greg Eisenhauer, Scott Klasky, Karsten Schwan, and Fang Zheng. 2010. DataStager: scalable data staging services for petascale applications. Cluster Computing 13, 3 (September 2010), 277--290.
2. Utkarsh Ayachit, Andrew Bauer, Earl P. N. Duque, Greg Eisenhauer, Nicola Ferrier, Junmin Gu, Kenneth E. Jansen, Burlen Loring, Zarija Lukić, Suresh Menon, Dmitriy Morozov, Patrick O'Leary, Reetesh Ranjan, Michel Rasquin, Christopher P. Stone, Venkat Vishwanath, Gunther H. Weber, Brad Whitlock, Matthew Wolf, K. John Wu, and E. Wes Bethel. 2016. Performance Analysis, Design Considerations, and Applications of Extreme-scale in Situ Infrastructures. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC'16). IEEE, 921--932.
3. Utkarsh Ayachit, Andrew Bauer, Berk Geveci, Patrick O'Leary, Kenneth Moreland, Nathan Fabian, and Jeffrey Mauldin. 2015. ParaView Catalyst: Enabling In Situ Data Analysis and Visualization. In First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV '15). ACM, New York, NY, USA, 25--29.
4. Utkarsh Ayachit, Brad Whitlock, Matthew Wolf, Burlen Loring, Berk Geveci, David Lonie, and E. Wes Bethel. 2016. The SENSEI Generic in Situ Interface. In Second Workshop on In Situ Infrastructures for Enabling Extreme-scale Analysis and Visualization (ISAV '16). IEEE, 40--44.
5. Hank Childs, Eric Brugger, Brad Whitlock, Jeremy Meredith, Sean Ahern, David Pugmire, Kathleen Biagas, Mark Miller, Cyrus Harrison, Gunther H. Weber, Hari Krishnan, Thomas Fogal, Allen Sanderson, Christoph Garth, E. Wes Bethel, David Camp, Oliver Rübel, Marc Durant, Jean M. Favre, and Paul Navrátil. 2012. VisIt: An End-User Tool For Visualizing and Analyzing Very Large Data. In High Performance Visualization-Enabling Extreme-Scale Scientific Insight. 357--372.
6. Jai Dayal, Drew Bratcher, Greg Eisenhauer, Karsten Schwan, Matthew Wolf, Xuechen Zhang, Hasan Abbasi, Scott Klasky, and Norbert Podhorszki. 2014. Flexpath: Type-Based Publish/Subscribe System for Large-Scale Science Analytics. In 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. IEEE, 246--255.
7. Thomas Fogal, Hank Childs, Siddharth Shankar, Jens Krüger, R Daniel Bergeron, and Philip Hatcher. 2010. Large Data Visualization on Distributed Memory Multi-GPU Clusters. In High Performance Graphics. The Eurographics Association, 57--66.
8. Steffen Frey and Thomas Ertl. 2011. Load Balancing Utilizing Data Redundancy in Distributed Volume Rendering. In Eurographics Symposium on Parallel Graphics and Visualization (EGPGV'11). The Eurographics Association, 10.
9. Patrick Gralka, Michael Becher, Matthias Braun, Florian Frieß, Christoph Müller, Tobias Rau, Karsten Schatz, Christoph Schulz, Michael Krone, Guido Reina, and Thomas Ertl. 2019. MegaMol - a comprehensive prototyping framework for visualizations. The European Physical Journal Special Topics 227, 14 (March 2019), 1817--1829.
10. Sebastian Grottel, Michael Krone, Cristoph Müller, Guido Reina, and Thomas Ertl. 2015. MegaMol - A Prototyping Framework for Particle-Based Visualization. IEEE Transactions on Visualization and Computer Graphics 21, 2 (February 2015), 201--214.
11. Matthias Heinen, Jadran Vrabec, and Johann Fischer. 2016. Communication: Evaporation: Influence of heat transport in the liquid on the interface temperature and the particle flux. The Journal of Chemical Physics 145, 8 (August 2016), 081101.
12. J. E. Jones. 1924. On the Determination of Molecular Fields. II. From the Equation of State of a Gas. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 106, 738 (October 1924), 463--477.
13. Mark Kim, Tom Evans, Scott Klasky, and David Pugmire. 2017. In Situ Visualization of Radiation Transport Geometry. In In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization (ISAV'17). ACM, New York, NY, USA, 7--11.
14. Matthew Larsen, James Ahrens, Utkarsh Ayachit, Eric Brugger, Hank Childs, Berk Geveci, and Cyrus Harrison. 2017. The ALPINE In Situ Infrastructure: Ascending from the Ashes of Strawman. In In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization (ISAV'17). ACM, New York, NY, USA, 42--46.
15. Nick Leaf, Bob Miller, and Kwan-Liu Ma. 2017. In Situ Video Encoding of Floating-Point Volume Data Using Special-Purpose Hardware for a Posteriori Rendering and Analysis. In IEEE 7th Symposium on Large Data Analysis and Visualization (LDAV'17). IEEE, 64--73.
16. Qing Liu, Jeremy Logan, Yuan Tian, Hasan Abbasi, Norbert Podhorszki, Jong Youl Choi, Scott Klasky, Roselyne Tchoua, Jay Lofstead, Ron Oldfield, Manish Parashar, Nagiza Samatova, Karsten Schwan, Arie Shoshani, Matthew Wolf, Kesheng Wu, and Weikuan Yu. 2014. Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworks. Concurrency and Computation: Practice and Experience 26, 7 (2014), 1453--1473.
17. Stéphane Marchesin, Catherine Mongenet, and Jean-Michel Dischler. 2006. Dynamic Load Balancing for Parallel Volume Rendering. In Eurographics Symposium on Parallel Graphics and Visualization (EGPGV'06). The Eurographics Association, 8.
18. Kenneth Moreland, Wesley Kendall, Tom Peterka, and Jian Huang. 2011. An Image Compositing Solution at Scale. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC'11). ACM, New York, NY, USA, Article 25, 10 pages.
19. Kenneth Moreland, Christopher Sewell, Will Usher, Li-ta Lo, Jeremy Meredith, David Pugmire, James Kress, Hendrik Schroots, Kwan-Liu Ma, Hank Childs, Matthew Larsen, Chun-Ming Chen, Robert Maynard, and Berk Geveci. 2016. VTK-m: Accelerating the Visualization Toolkit for Massively Threaded Architectures. IEEE Computer Graphics and Applications 36, 3 (May 2016), 48--58.
20. Christoph Niethammer, Stefan Becker, Martin Bernreuther, Martin Buchholz, Wolfgang Eckhardt, Alexander Heinecke, Stephan Werth, Hans-Joachim Bungartz, Colin W. Glass, Hans Hasse, Jadran Vrabec, and Martin Horsch. 2014. ls1 mardyn: The Massively Parallel Molecular Dynamics Code for Large Systems. Journal of Chemical Theory and Computation 10, 10 (2014), 4455--4464.
21. Patrick O'Leary, James Ahrens, Sébastien Jourdain, Scott Wittenburg, David H. Rogers, and Mark Petersen. 2016. Cinema image-based in situ analysis and visualization of MPAS-ocean simulations. Parallel Comput. 55 (July 2016), 43--48.
22. Tobias Rau, Michael Krone, Guido Reina, and Thomas Ertl. 2017. Challenges and Opportunities using Software-Defined Visualization in MegaMol. In Workshop on Visual Analytics, Information Visualization and Scientific Visualization (WVIS) in the 30th Conference on Graphics, Patterns and Images (SIBGRAPI'17). http://sibgrapi2017.ic.uff.br/e-proceedings/assets/papers/WVIS/WVIS2.pdf
23. Will Usher, Silvio Rizzi, Ingo Wald, Jefferson Amstutz, Joseph Insley, Venkatram Vishwanath, Nicola Ferrier, Michael E. Papka, and Valerio Pascucci. 2018. libIS: A Lightweight Library for Flexible In Transit Visualization. In Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV'18). ACM, New York, NY, USA, 33--38.
24. Venkatram Vishwanath, Mark Hereld, Vitali Morozov, and Michael E. Papka. 2011. Topology-aware data movement and staging for I/O acceleration on Blue Gene/P supercomputing systems. In International Conference for High Performance Computing, Networking, Storage and Analysis (SC'11). ACM, New York, NY, USA, Article 19, 11 pages.
25. Jadran Vrabec, Martin Bernreuther, Hans-Joachim Bungartz, Wei-Lin Chen, Wilfried Cordes, Robin Fingerhut, Colin W. Glass, Jürgen Gmehling, René Hamburger, Manfred Heilig, Matthias Heinen, Martin T. Horsch, Chieh-Ming Hsieh, Marco Hülsmann, Philip Jäger, Peter Klein, Sandra Knauer, Thorsten Köddermann, Andreas Köster, Kai Langenbach, Shiang-Tai Lin, Philipp Neumann, Jürgen Rarey, Dirk Reith, Gábor Rutkai, Michael Schappals, Martin Schenk, Andre Schedemann, Mandes Schönherr, Steffen Seckler, Simon Stephan, Katrin Stöbener, Nikola Tchipev, Amer Wafai, Stephan Werth, and Hans Hasse. 2018. SkaSim - Skalierbare HPC-Software für molekulare Simulationen in der chemischen Industrie. Chemie Ingenieur Technik 90, 3 (March 2018), 295--306.
26. I Wald, GP Johnson, J Amstutz, C Brownlee, A Knoll, J Jeffers, J Günther, and P Navrátil. 2017. OSPRay - A CPU Ray Tracing Framework for Scientific Visualization. IEEE Transactions on Visualization and Computer Graphics 23, 1 (January 2017), 931--940.
27. Brad Whitlock, Jean M. Favre, and Jeremy S. Meredith. 2011. Parallel In Situ Coupling of Simulation with a Fully Featured Visualization System. In Eurographics Symposium on Parallel Graphics and Visualization (EGPGV'11). The Eurographics Association, 9.
28. Yucong (Chris) Ye, Tyson Neuroth, Franz Sauer, Kwan-Liu Ma, Giulio Borghesi, Aditya Konduri, Hemanth Kolla, and Jacqueline Chen. 2016. In Situ Generated Probability Distribution Functions for Interactive Post Hoc Visualization and Analysis. In Symposium on Large Data Analysis and Visualization (LDAV'16). IEEE, 65--74.
29. ZeroMQ. 2019. ZeroMQ Messaging Library. Retrieved 2019-10-20 from https://zeromq.org/
30. Fang Zheng, Hongbo Zou, Greg Eisenhauer, Karsten Schwan, Matthew Wolf, Jai Dayal, Tuan-Anh Nguyen, Jianting Cao, Hasan Abbasi, Scott Klasky, Norbert Podhorszki, and Hongfeng Yu. 2013. FlexIO: I/O Middleware for Location-Flexible Scientific Data Analytics. In International Symposium on Parallel and Distributed Processing (IPDPS'13). IEEE, 320--331.

Index Terms

• Computing methodologies

  • Computer graphics

    • Rendering

      • Ray tracing

  • Modeling and simulation

    • Simulation evaluation

    • Simulation types and techniques

      • Molecular simulation

• Human-centered computing

  • Visualization

    • Visualization application domains

      • Scientific visualization