Virtual reality in software engineering: affordances, applications, and challenges

ABSTRACT

Software engineers primarily interact with source code using a keyboard and mouse, and typically view software on a small number of 2D monitors. This interaction paradigm does not take advantage of many affordances of natural human movement and perception. Virtual reality (VR) can use these affordances more fully than existing developer environments to enable new creative opportunities and potentially result in higher productivity, lower learning curves, and increased user satisfaction. This paper describes the affordances offered by VR; demonstrates the benefits of VR and software engineering in prototypes for live coding and code review; and discusses future work, open questions, and the challenges of VR.

References

1. A. Agarawala and R. Balakrishnan. Keepin'it real: pushing the desktop metaphor with physics, piles and the pen. In Proc. of the SIGCHI conference on HFCS, pages 1283--1292. ACM, 2006.
2. A. Bacchelli and C. Bird. Expectations, outcomes, and challenges of modern code review. In Proc. of ICSE, pages 712--721. IEEE Press, 2013.
3. A. Bragdon, S. P. Reiss, R. Zeleznik, S. Karumuri, W. Cheung, J. Kaplan, C. Coleman, F. Adeputra, and J. J. LaViola, Jr. Code bubbles: Rethinking the user interface paradigm of integrated development environments. In Proc. of the 32nd ACM/IEEE ICSE - Volume 1, ICSE '10, pages 455--464, New York, NY, USA, 2010. ACM.
4. M. M. Burnett, J. W. Atwood Jr, and Z. T. Welch. Implementing level 4 liveness in declarative visual programming languages. In Visual Languages, 1998. Proceedings. 1998 IEEE Symposium on, pages 126--133. IEEE, 1998.
5. D. Delimarschi, G. Swartzendruber, and H. Kagdi. Enabling integrated development environments with natural user interface interactions. ICPC 2014, pages 126--129, New York, NY, USA, 2014. ACM.
6. R. DeLine and K. Rowan. Code canvas: zooming towards better development environments. In Proc. of the 32nd ACM/IEEE ICSE-Volume 2, pages 207--210. ACM, 2010.
7. A. D. Ekstrom, M. J. Kahana, J. B. Caplan, T. A. Fields, E. A. Isham, E. L. Newman, and I. Fried. Cellular networks underlying human spatial navigation. Nature, 425(6954):184--188, Sept. 2003.
8. R. HELD and A. HEIN. Movement-produced stimulation in the development of visually guided behavior. Journal of comparative and physiological psychology, 56:872--876, Oct. 1963.
9. A. Z. Henley and S. D. Fleming. The patchworks code editor: Toward faster navigation with less code arranging and fewer navigation mistakes. In Proc. of the SIGCHI Conference on HFCS, CHI '14, pages 2511--2520, New York, NY, USA, 2014. ACM.
10. A. J. Ko, B. A. Myers, and H. H. Aung. Six learning barriers in end-user programming systems. In Visual Languages and Human Centric Computing, 2004 IEEE Symposium on, pages 199--206. IEEE, 2004.
11. A. J. Ko, B. A. Myers, M. J. Coblenz, and H. H. Aung. An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. IEEE Trans. Softw. Eng., 32(12):971--987, Dec. 2006.
12. A. Kuhn, D. Erni, and O. Nierstrasz. Embedding spatial software visualization in the ide: An exploratory study. In Proc. of the 5th International Symposium on Software Visualization, SOFTVIS '10, pages 113--122, New York, NY, USA, 2010. ACM.
13. R. T. K. R. Maalej Walid, Tiarks Rebecca. ACM Transactions in Software Engineering and Methodology, 23(4):31:1--31:37, 2014.
14. D. A. Norman. The psychology of everyday things. Basic books, 1988.
15. J. M. Noyes and K. J. Garland. Computer- vs. paper-based tasks: are they equivalent? Ergonomics, 51(9):1352--1375, Sept. 2008.
16. M. Oppezzo and D. L. Schwartz. Give your ideas some legs: The positive effect of walking on creative thinking. Journal of experimental psychology. Learning, memory, and cognition, Apr. 2014.
17. C. Parnin and S. Rugaber. Programmer information needs after memory failure. ICPC 2012, pages 123--132, June 2012.
18. G. Robertson, M. Czerwinski, K. Larson, D. C. Robbins, D. Thiel, and M. van Dantzich. Data mountain: Using spatial memory for document management. UIST '98, pages 153--162, New York, NY, USA. ACM.
19. B. Shneiderman. The eyes have it: A task by data type taxonomy for information visualizations. In Visual Languages, 1996. Proc., IEEE Symposium on, pages 336--343. IEEE, 1996.
20. M. Slater. Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philosophical Transactions of the Royal Society B: Biological Sciences, 364(1535):3549--3557, 2009.
21. A. R. Teyseyre and M. R. Campo. An overview of 3d software visualization. Visualization and Computer Graphics, IEEE Transactions on, 15(1):87--105, 2009.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Virtual reality

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Mixed / augmented reality

      • Virtual reality

• Software and its engineering