Large data visualization on distributed memory multi-GPU clusters

ABSTRACT

Data sets of immense size are regularly generated on large scale computing resources. Even among more traditional methods for acquisition of volume data, such as MRI and CT scanners, data which is too large to be effectively visualized on standard workstations is now commonplace.

One solution to this problem is to employ a 'visualization cluster,' a small to medium scale cluster dedicated to performing visualization and analysis of massive data sets generated on larger scale supercomputers. These clusters are designed to fit a different need than traditional supercomputers, and therefore their design mandates different hardware choices, such as increased memory, and more recently, graphics processing units (GPUs). While there has been much previous work on distributed memory visualization as well as GPU visualization, there is a relative dearth of algorithms which effectively use GPUs at a large scale in a distributed memory environment. In this work, we study a common visualization technique in a GPU-accelerated, distributed memory setting, and present performance characteristics when scaling to extremely large data sets.

References

1. {CBB*05} Childs H., Brugger E., Bonnell K., Meredith J., Miller M., Whitlock B., Max N.: A Contract Based System For Large Data Visualization. In Proceedings of IEEE Visualization 2005 (2005). http://www.idav.ucdavis.edu/func/return_pdf?pub_id=890.
2. {CCF94} Cabral B., Cam N., Foran J.: Accelerated volume rendering and tomographic reconstruction using texture mapping hardware. In VVS '94: Proceedings of the 1994 symposium on Volume visualization (New York, NY, USA, 1994), ACM, pp. 91--98. http://doi.acm.org/10.1145/197938.197972.
3. {CDM06} Childs H., Duchaineau M., Ma K.-L.: A scalable, hybrid scheme for volume rendering massive data sets. In Proceedings of Eurographics Symposium on Parallel Graphics and Visualization (May 2006), pp. 153--162. http://www.idav.ucdavis.edu/publications/print_pub?pub_id=892.
4. {CN94} Cullip T. J., Neumann U.: Accelerating Volume Reconstruction With 3D Texture Hardware. Tech. Rep. TR93--027, University of North Carolina at Chapel Hill, 1994. http://graphics.usc.edu/cgit/pdf/papers/Volume_textures_93.pdf.
5. {DCH88} Drebin R. A., Carpenter L., Hanrahan P.: Volume Rendering. In SIGGRAPH '88: Proceedings of the 15th annual conference on Computer graphics and interactive techniques (New York, NY, USA, 1988), ACM, pp. 65--74. http://doi.acm.org/10.1145/54852.378484.
6. {EP07} Eilemann S., Pajarola R.: Direct Send Compositing for Parallel Sort-Last Rendering. In Proceedings of the Eurographics Symposium on Parallel Graphics and Visualization (2007), pp. 29--36. http://doi.acm.org/10.1145/1508044.1508083.
7. {HBC10} Howison M., Bethel E. W., Childs H.: MPI-hybrid Parallelism for Volume Rendering on Large, Multi-core Systems. In Eurographics Symposium on Parallel Graphics and Visualization (EGPGV) (Norrköping, Sweden, May 2010). LBNL-3297E.
8. {HHN*02} Humphreys G., Houston M., Ng R., Frank R., Ahern S., Kirchner P. D., Klosowski J. T.: Chromium: A Stream-Processing Framework for Interactive Rendering on Clusters. In SIGGRAPH '02: Proceedings of the 29th annual conference on Computer graphics and interactive techniques (New York, NY, USA, 2002), ACM Press, pp. 693--702. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.19.7869.
9. {Hsu93} Hsu W. M.: Segmented Ray Casting for Data Parallel Volume Rendering. In PRS '93: Proceedings of the 1993 Symposium on Parallel Rendering (New York, NY, USA, 1993), pp. 7--14. http://doi.acm.org/10.1145/166181.166182.
10. {KW03} Krüger J., Westermann R.: Acceleration Techniques for GPU-based Volume Rendering. In Proceedings IEEE Visualization 2003 (2003). http://wwwcg.in.tum.de/Research/data/vis03-rc.pdf.
11. {Lev90} Levoy M.: Efficient Ray Tracing of Volume Data. ACM Trans. Graph. 9, 3 (1990), 245--261. http://doi.acm.org/10.1145/78964.78965.
12. {Max95} Max N.: Optical Models for Direct Volume Rendering. IEEE Transactions on Visualization and Computer Graphics 1, 2 (1995), 99--108. http://www.llnl.gov/graphics/docs/OpticalModelsLong.pdf.
13. {MCEF94} Molnar S., Cox M., Ellsworth D., Fuchs H.: A Sorting Classification of Parallel Rendering. IEEE Comput. Graph. Appl. 14, 4 (1994), 23--32. http://doi.acm.org/10.1145/1508044.1508079.
14. {MMD06} Marchesin S., Mongenet C., Dischler J.-M.: Dynamic Load Balancing for Parallel Volume Rendering. In 6th Eurographics Symposium on Parallel Graphics and Visualization (May 2006), pp. 43--50. http://people.freedesktop.org/~marcheu/egpgv06-loadbalancing.pdf.
15. {MMD08} Marchesin S., Mongenet C., Dischler J.-M.: Multi-GPU Sort-Last Volume Visualization. In EG Symposium on Parallel Graphics and Visualization (EGPGV'08), Eurographics (April 2008). http://icps.u-strasbg.fr/~marchesin/egpgv08-multigpu.pdf.
16. {MPHK93} Ma K. L., Painter J. S., Hansen C. D., Krogh M. F.: A Data Distributed, Parallel Algorithm for Ray-Traced Volume Rendering. In PRS '93: Proceedings of the 1993 symposium on Parallel Rendering (New York, NY, USA, 1993), ACM, pp. 15--22. http://doi.acm.org/10.1145/166181.166183.
17. {MPHK94} Ma K.-L., Painter J. S., Hansen C. D., Krogh M. F.: Parallel Volume Rendering Using Binary-Swap Compositing. IEEE Comput. Graph. Appl. 14, 4 (1994), 59--68. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.104.3283.
18. {MSE06} Müller C., Strengert M., Ertl T.: Optimized Volume Raycasting for Graphics-Hardware-based Cluster Systems. In Eurographics Symposium on Parallel Graphics and Visualization (EGPGV06) (2006), Eurographics Association, pp. 59--66. http://www.vis.uni-stuttgart.de/ger/research/pub/pub2006/egpgv06-mueller.pdf.
19. {MWP01} Moreland K., Wylie B. N., Pavlakos C. J.: Sort-Last Parallel Rendering for Viewing Extremely Large Data Sets on Tile Displays. In IEEE Symposium on Parallel and Large-Data Visualization and Graphics (2001), pp. 85--92. https://cfwebprod.sandia.gov/cfdocs/CCIM/docs/PVG2001.pdf.
20. {PD84} Porter T., Duff T.: Compositing Digital Images. In SIGGRAPH '84: Proceedings of the 11th annual conference on Computer graphics and interactive techniques (New York, NY, USA, 1984), ACM, pp. 253--259. http://doi.acm.org/10.1145/964965.808606.
21. {PYR*09} Peterka T., Yu H., Ross R., Ma K.-L., Latham R.: End-to-End Study of Parallel Volume Rendering on the IBM Blue Gene/P. In Proceedings of the ICPP'09 Conference (September 2009). http://vis.cs.ucdavis.edu/Ultravis/papers/129_peterka-icpp09-finalpaper.pdf.
22. {PYRM08} Peterka T., Yu H., Ross R., Ma K.-L.: Parallel volume rendering on the ibm blue gene/p. In Proceedings of Eurographics Parallel Graphics and Visualization Symposium (EGPGV 2008) (April 2008), pp. 73--80. http://vis.cs.ucdavis.edu/papers/EGPGV_08.pdf.
23. {SMW*04} Strengert M., Magallón M., Weiskopf D., Guthe S., Ertl T.: Hierarchical visualization and compression of large volume datasets using gpu clusters. In In Eurographics Symposium on Parallel Graphics and Visualization (EGPGV04) (2004 (2004), pp. 41--48.
24. {WE98} Westermann R., Ertl T.: Efficiently Using Graphics Hardware in Volume Rendering Applications. In ACM SIGGRAPH 1998 (1998). http://doi.acm.org/10.1145/280814.280860.

Index Terms

• Applied computing

  • Document management and text processing

    • Document capture

      • Graphics recognition and interpretation

• Computer systems organization

  • Dependable and fault-tolerant systems and networks

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Graphics processors

• General and reference

  • Cross-computing tools and techniques

    • Performance

• Networks

  • Network performance evaluation