Clusters, Trends, and Outliers: How Immersive Technologies Can Facilitate the Collaborative Analysis of Multidimensional Data

ABSTRACT

Immersive technologies such as augmented reality devices are opening up a new design space for the visual analysis of data. This paper studies the potential of an augmented reality environment for the purpose of collaborative analysis of multidimensional, abstract data. We present ART, a collaborative analysis tool to visualize multidimensional data in augmented reality using an interactive, 3D parallel coordinates visualization. The visualization is anchored to a touch-sensitive tabletop, benefiting from well-established interaction techniques. The results of group-based, expert walkthroughs show that ART can facilitate immersion in the data, a fluid analysis process, and collaboration. Based on the results, we provide a set of guidelines and discuss future research areas to foster the development of immersive technologies as tools for the collaborative analysis of multidimensional data.

References

1. Daniel Belcher, Mark Billinghurst, S.E. Hayes, and Randy Stiles. 2003. Using Augmented Reality for Visualizing Complex Graphs in Three Dimensions. In Proceedings of the IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '03). IEEE Computer Society, Washington, DC, USA, 1--9.
2. Hrvoje Benko and Steven Feiner. 2007. Balloon Selection: A Multi-Finger Technique for Accurate Low-Fatigue 3D Selection. In Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI '07). IEEE Computer Society, Washington, DC, USA, 79--86.
3. Hrvoje Benko, Edward W. Ishak, and Steven Feiner. 2004. Collaborative Mixed Reality Visualization of an Archaeological Excavation. In Proceedings of the IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '04). IEEE Computer Society, Washington, DC, USA, 132--140.
4. Hrvoje Benko, Edward W. Ishak, and Steven Feiner. 2005. Cross-Dimensional Gestural Interaction Techniques for Hybrid Immersive Environments. In Proceedings of the IEEE Conference on Virtual Reality (VR '05). IEEE Computer Society, Washington, DC, USA, 209--216.
5. Rebecca Bennett, David J Zielinski, and Regis Kopper. 2014. Comparison of Interactive Environments for the Archaeological Exploration of 3D Landscape Data. In Proceedings of the IEEE VIS International Workshop on 3DVis (3DVis '14). IEEE Computer Society, Washington, DC, USA, 67--71.
6. Alberto Betella, Rodrigo Carvalho, Jesus Sanchez-Palencia, Ulysses Bernardet, and Paul F. M. J. Verschure. 2012. Embodied Interaction with Complex Neuronal Data in Mixed-Reality. In Proceedings of the Virtual Reality International Conference (VRIC '12). ACM Press, New York, NY, USA, 1.
7. Anastasia Bezerianos, Fanny Chevalier, Pierre Dragicevic, Niklas Elmqvist, and Jean-Daniel Fekete. 2010. GraphDice: A System for Exploring Multivariate Social Networks. Computer Graphics Forum 29, 3 (2010), 863--872.
8. Mark Billinghurst and Hirokazu Kato. 1999. Collaborative Mixed Reality. In Proceedings of the International Symposium on Mixed Reality (ISMR '99). Mixed Reality - Merging Real and Virtual Worlds. Springer, Berlin, DE, 261--284.
9. Doug A. Bowman, Ernst Kruijff, Joseph J. Laviola, and Ivan Poupyrev. 2004. 3D User Interfaces: Theory and Practice. Addison-Wesley, Redwood City, CA, USA. 478 pages.
10. Gerd Bruder, Frank Steinicke, and Wolfgang Stuerzlinger. 2013. Touching the Void Revisited: Analyses of Touch Behavior on and above Tabletop Surfaces. In Proceedings of the IFIP Conference on Human-Computer Interaction (INTERACT '13). Springer, Berlin, DE, 278--296.
11. Andreas Butz, Thomas Höllerer, Steven Feiner, Blair MacIntyre, and Clifford Beshers. 1999. Enveloping Users and Computers in a Collaborative 3D Augmented Reality. In Proceedings of the IEEE/ACM International Workshop on Augmented Reality (IWAR '99). IEEE Computer Society, Washington, DC, USA, 35--44.
12. Tom Chandler, Maxime Cordeil, Tobias Czauderna, Tim Dwyer, Jaroslaw Glowacki, Cagatay Goncu, Matthias Klapperstueck, Karsten Klein, Kim Marriott, Falk Schreiber, and Elliot Wilson. 2015. Immersive Analytics. In Proceedings of the IEEE International Symposium on Big Data Visual Analytics (BDVA '15). 1--8.
13. Dane Coffey, Nicholas Malbraaten, Trung Le, Iman Borazjani, Fotis Sotiropoulos, and Daniel F Keefe. 2011. Slice WIM: A Multi-Surface, Multi-Touch Interface for Overview+Detail Exploration of Volume Datasets in Virtual Reality. In Proceeding of the Symposium on Interactive 3D Graphics and Games (I3D '11). ACM Press, New York, NY, USA, 191--198.
14. Dane Coffey, Nicholas Malbraaten, Trung Bao Le, Iman Borazjani, Fotis Sotiropoulos, Arthur G Erdman, and Daniel F Keefe. 2012. Interactive Slice WIM: Navigating and Interrogating Volume Data Sets Using a Multisurface, Multitouch VR Interface. IEEE Transactions on Visualization and Computer Graphics 18, 10 (Oct 2012), 1614--1626.
15. Christopher Collins and Sheelagh Carpendale. 2007. VisLink: Revealing Relationships Amongst Visualizations. IEEE Transactions on Visualization and Computer Graphics 13, 6 (2007), 1192--1199.
16. Maxime Cordeil, Andrew Cunningham, Tim Dwyer, Bruce H. Thomas, and Kim Marriott. 2017a. ImAxes: Immersive Axes as Embodied Affordances for Interactive Multivariate Data Visualisation. In Proceedings of the ACM Symposium on User Interface and Software Technology (UIST '17). ACM Press, New York, NY, USA, 71--83.
17. Maxime Cordeil, Tim Dwyer, Karsten Klein, Bireswar Laha, Kim Marriott, and Bruce H. Thomas. 2017b. Immersive Collaborative Analysis of Network Connectivity: CAVE-style or Head-Mounted Display? IEEE Transactions on Visualization and Computer Graphics 23, 1 (Jan 2017), 441--450.
18. Carolina Cruz-Neira, Daniel J. Sandin, and Thomas A. DeFanti. 1993. Surround-Screen Projection-Based Virtual Reality : The Design and Implementation of the CAVE. In Proceedings of the Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '93). ACM Press, New York, NY, USA, 135--142.
19. Ciro Donalek, S. G. Djorgovski, Alex Cioc, Anwell Wang, Jerry Zhang, Elizabeth Lawler, Stacy Yeh, Ashish Mahabal, Matthew Graham, Andrew Drake, Scott Davidoff, Jeffrey S. Norris, and Giuseppe Longo. 2014. Immersive and Collaborative Data Visualization using Virtual Reality Platforms. In Proceedings of the IEEE International Conference on Big Data. IEEE Computer Society, Washington, DC, USA, 609--614.
20. Daniel Drochtert and Christian Geiger. 2015. Collaborative Magic Lens Graph Exploration. In Proceedings of the SIGGRAPH Asia Symposium On Mobile Graphics And Interactive Applications (SA '15). ACM Press, New York, NY, USA, 1--3.
21. Niklas Elmqvist, Pierre Dragicevic, and Jean Daniel Fekete. 2008. Rolling the Dice: Multidimensional Visual Exploration using Scatterplot Matrix Navigation. IEEE Transactions on Visualization and Computer Graphics 14, 6 (2008), 1141--1148.
22. Niklas Elmqvist, Andrew Vande Moere, Hans-Christian Jetter, Daniel Cernea, Harald Reiterer, and T. J. Jankun-Kelly. 2011. Fluid Interaction for Information Visualization. Information Visualization - Special Issue on State of the Field and New Research Directions 10, 4 (2011), 327--340.
23. Elena Fanea, Sheelagh Carpendale, and Tobias Isenberg. 2005. An Interactive 3D Integration of Parallel Coordinates and Star Glyphs. In Proceedings of the IEEE Symposium on Information Visualization (INFOVIS '05). IEEE Computer Society, Washington, DC, USA, 149--156.
24. Edwin B. Fisher, Marian L. Fitzgibbon, Russell E. Glasgow, Debra Haire-Joshu, Laura L. Hayman, Robert M. Kaplan, Marilyn S. Nanney, and Judith K. Ockene. 2011. Behavior Matters. American Journal of Preventive Medicine 40, 5 (2011), e15 -- e30.
25. Michael Friendly. 2009. Milestones in the history of thematic cartography, statistical graphics, and data visualization. Website. (2009). Retrieved August 28, 2017 from http://www.datavis.ca/milestones/.
26. Anton Fuhrmann, Gerd Hesina, François Faure, and Michael Gervautz. 1999. Occlusion in collaborative augmented environments. Computers & Graphics 23, 6 (Dec 1999), 809--819.
27. Anton Fuhrmann, Helwig Löffelmann, Dieter Schmalstieg, and Michael Gervautz. 1998. Collaborative Visualization in Augmented Reality. IEEE Computer Graphics and Applications 18, 4 (1998), 54--59.
28. Martin Hachet, Benoît Bossavit, Aurélie Cohé, and Jean-Baptiste de la Rivière. 2011. Toucheo: Multitouch and Stereo Combined in a Seamless Workspace. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST '11). ACM Press, New York, NY, USA, 587--592.
29. Jeffrey Heer and Maneesh Agrawala. 2008. Design Considerations for Collaborative Visual Analytics. Information Visualization - Special issue on Visual Analytics Science and Technology 7, 1 (Mar 2008), 49--62.
30. Kristin E. Heron and Joshua M. Smyth. 2010. Ecological Momentary Interventions: Incorporating Mobile Technology Into Psychosocial and Health Behavior Treatments. British Journal of Health Psychology 15, 1 (2010), 1--39.
31. Petra Isenberg, Niklas Elmqvist, Jean Scholtz, Daniel Cernea, Kwan-Liu Ma, and Hans Hagen. 2011. Collaborative visualization: Definition, challenges, and research agenda. Information Visualization - Special issue on State of the Field and New Research Directions 10, 4 (Oct 2011), 310--326.
32. Bireswar Laha, Doug A. Bowman, and James D. Schiffbauer. 2013. Validation of the MR Simulation Approach for Evaluating the Effects of Immersion on Visual Analysis of Volume Data. IEEE Transactions on Visualization and Computer Graphics 19, 4 (Apr 2013), 529--538.
33. Alexander Lex, Marc Streit, Ernst Kruijff, and Dieter Schmalstieg. 2010. Caleydo: Design and Evaluation of a Visual Analysis Framework for Gene Expression Data in its Biological Context. In Proceedings of the IEEE Pacific Visualization Symposium (PacificVis '11). IEEE Computer Society, Washington, DC, USA, 57--64.
34. David López, Lora Oehlberg, Candemir Doger, and Tobias Isenberg. 2016. Towards An Understanding of Mobile Touch Navigation in a Stereoscopic Viewing Environment for 3D Data Exploration. IEEE Transactions on Visualization and Computer Graphics 22, 5 (May 2016), 1--13.
35. Bianchi Serique Meiguins, Ricardo Melo Casseb do Carmo, Leonardo Almeida, Aruanda Simões Gonçalves, Sérgio Clayton V. Pinheiro, Marcelo de Brito Garcia, and Paulo Igor Alves Godinho. 2006. Multidimensional Information Visualization Using Augmented Reality. In Proceedings of the ACM International Conference on Virtual Reality Continuum and its Applications (VRCIA '06). ACM Press, New York, NY, USA, 14--17.
36. Jens Müller, Simon Butscher, and Harald Reiterer. 2016. Immersive Analysis of Health-Related Data with Mixed Reality Interfaces: Potentials and Open Question. In Proceedings of the ACM Companion on Interactive Surfaces and Spaces (ISS Companion '16). ACM Press, New York, NY, USA, 71--76.
37. Michael Narayan, Leo Waugh, Xiaoyu Zhang, Pradyut Bafna, and Doug Bowman. 2005. Quantifying the Benefits of Immersion for Collaboration in Virtual Environments. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '05). ACM Press, New York, NY, USA, 78--81.
38. Huyen Nguyen, Peter Marendy, and Ulrich Engelke. 2016. Collaborative Framework Design for Immersive Analytics. In Proceedings of the IEEE International Symposium on Big Data Visual Analytics (BDVA '16). IEEE Computer Society, Washington, DC, USA, 23--30.
39. Dheva Raja, Doug A. Bowman, John Lucas, and Chris North. 2004. Exploring the Benefits of Immersion in Abstract Information Visualization. In Proceedings of the Immersive Projection Technology Workshop.
40. Gabriel Robles-De-La-Torre. 2006. The Importance of the Sense of Touch in Virtual and Real Environments. IEEE Multimedia 13, 3 (Jul 2006), 24--30.
41. René Rosenbaum, Jeremy Bottleson, Zhuiguang Liu, and Bernd Hamann. 2011. Involve Me and I Will Understand!-Abstract Data Visualization in Immersive Environments. In Proceedings of the International Symposium on Visual Computing: Advances in Visual Computing (ISVC '11), Vol. 6938 LNCS. 530--540.
42. Dieter Schmalstieg, Anton Fuhrmann, Gerd Hesina, Zsolt Szalavári, L. Miguel Encarnação, Michael Gervautz, and Werner Purgathofer. 2002. The Studierstube Augmented Reality Project. Presence: Teleoperators and Virtual Environments 11, 1 (Feb 2002), 33--54.
43. Neil G. Smith, Kyle Knabb, Connor DeFanti, Philip Weber, Jurgen Schulze, Andrew Prudhomme, Falko Kuester, Thomas E. Levy, and Thomas A. DeFanti. 2013. ArtifactVis2: Managing real-time archaeological data in immersive 3D environments. In Proceedings of the Digital Heritage International Congress (DigitalHeritage '13). The Eurographics Association, 363--370.
44. Maurício Sousa, Daniel Mendes, Soraia Paulo, Nuno Matela, Joaquim Jorge, and Daniel Simões Lopes. 2017. VRRRRoom: Virtual Reality for Radiologists in the Reading Room. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '17). ACM Press, New York, NY, USA, 4057--4062.
45. Sven Strothoff, Dimitar Valkov, and Klaus Hinrichs. 2011. Triangle Cursor: Interactions With Objects Above the Tabletop. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS '11). ACM Press, New York, NY, USA, 111--119.
46. Zsolt Szalavári, Dieter Schmalstieg, Anton Fuhrmann, and Michael Gervautz. 1998. "Studierstube": An Environment for Collaboration in Augmented Reality. Virtual Reality 3, 1 (Mar 1998), 37--48.
47. Unity Technologies. 2017. Unity - Game Engine. Website. (Aug 2017). Retrieved August 28, 2017 from https://unity3d.com/.
48. Christophe Viau, Michael J. McGuffin, Yves Chiricota, and Igor Jurisica. 2010. The FlowVizMenu and Parallel Scatterplot Matrix: Hybrid Multidimensional Visualizations for Network Exploration. IEEE Transactions on Visualization and Computer Graphics 16, 6 (Nov 2010), 1100--1108.
49. Luc Vlaming, Christopher Collins, Mark Hancock, Miguel Nacenta, Tobias Isenberg, and Sheelagh Carpendale. 2010. Integrating 2D Mouse Emulation with 3D Manipulation for Visualizations on a Multi-Touch Table. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (ITS '10). ACM Press, New York, NY, USA, 221--230.
50. Colin Ware and Glenn Franck. 1994. Viewing a Graph in a Virtual Reality Display is Three Times as Good as a 2D Diagram. In Proceedings of the IEEE Symposium on Visual Languages (VL '94). IEEE Computer Society, Washington, DC, USA, 182--183.
51. Song Zhang, Çağatay Demiralp, Daniel F. Keefe, Marco DaSilva, David H. Laidlaw, Benjamin D. Greenberg, Peter J. Basser, Carlo Pierpaoli, Ennio Antonio Chiocca, and Thomas S. Deisboeck. 2001. An Immersive Virtual Environment for DT-MRI Volume Visualization Applications: a Case Study. In Proceedings of the Conference on Visualization (VIS '01). IEEE Computer Society, Washington, DC, USA, 437--440.
52. Sean Ziegeler, R.J. Moorhead, P.J. Croft, and Duanjun Lu. 2001. The MetVR Case Study: Meteorological Visualization in an Immersive Virtual Environment. In Proceedings of the Conference on Visualization (VIS '01). IEEE Computer Society, Washington, DC, USA, 489--596.

Index Terms

• Information systems

  • Information systems applications

    • Multimedia information systems