Automatic generation of semantic icon encodings for visualizations

ABSTRACT

Authors use icon encodings to indicate the semantics of categorical information in visualizations. The default icon libraries found in visualization tools often do not match the semantics of the data. Users often manually search for or create icons that are more semantically meaningful. This process can hinder the flow of visual analysis, especially when the amount of data is large, leading to a suboptimal user experience. We propose a technique for automatically generating semantically relevant icon encodings for categorical dimensions of data points. The algorithm employs natural language processing in order to find relevant imagery from the Internet. We evaluate our approach on Mechanical Turk by generating large libraries of icons using Tableau Public workbooks that represent real analytical effort by people out in the world. Our results show that the automatic algorithm does nearly as well as the manually created icons, and particularly has higher user satisfaction for larger cardinalities of data.

References

1. Björk, S., Holmquist, L. E., Redström, J., Bretan, I., Danielsson, R., Karlgren, J., and Franzén, K. West: a web browser for small terminals. In Proceedings of the 12th Annual ACM Symposium on User Interface Software and Technology (UIST), ACM (New York, NY, USA, 1999), 187--196.
2. Blattner, M. M., Sumikawa, D. A., and Greenberg, R. M. Earcons and icons: Their structure and common design principles (abstract only). SIGCHI Bulletin 21, 1 (Aug. 1989), 123--124.
3. Cleveland, W. S., and McGill, R. Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of the American Statistical Association 79, 387 (1984), pp. 531--554.
4. Coren, S., and Girgus, J. Principles of perceptual organization and spatial distortion: The gestalt illusions. In Journal of Experimental Psychology: Human Perception and Performance, vol. 6, Morgan Kaufmann (1980), 404--412.
5. Familant, M. E., and Detweiler, M. C. Iconic reference: Evolving perspectives and an organizing framework. vol. 39, Academic Press Ltd. (London, UK, UK, Nov. 1993), 705--728.
6. Fellbaum, C., Ed. WordNet: An Electronic Lexical Database. MIT Press, 1998.
7. Gittins, D. Icon-based human-computer interaction. vol. 24, Academic Press Ltd. (London, UK, UK, June 1986), 519--543.
8. Healey, C. G., Booth, K. S., and Enns, J. T. Visualizing real-time multivariate data using preattentive processing. ACM Transactions on Modeling and Computer Simulation 5, 3 (July 1995), 190--221.
9. Henry, T. R., and Hudson, S. E. Multidimensional icons. vol. 9, ACM (New York, NY, USA, 1990), 133--137.
10. Kosslyn, S. M. Graphics and human information processing: A review of five books. vol. 80, Journal of the American Statistical Association (1985), 499--512.
11. Lewis, J. P., Rosenholtz, R., Fong, N., and Neumann, U. Visualids: Automatic distinctive icons for desktop interfaces. In ACM Special Interest Group on Graphics and Interactive Techniques (SIGGRAPH), ACM (New York, NY, USA, 2004), 416--423.
12. Li, Y., Bandar, Z. A., and McLean, D. An approach for measuring semantic similarity between words using multiple information sources. IEEE Transactions on Knowledge and Data Engineering 15, 4 (July 2003), 871--882.
13. Mackinlay, J. Automating the design of graphical presentations of relational information. ACM Transactions on Graphics (TOG) 5, 2 (Apr. 1986), 110--141.
14. Marcus, A. Icons, symbols, and signs: Visible languages to facilitate communication. Interactions 10, 3 (May 2003), 37--43.
15. Nowell, L. T. Graphical encoding for information visualization: Using icon color, shape, and size to convey nominal and quantitative data. Virginia Polytechnic Institute and State University (1997).
16. Resnik, P. Using information content to evaluate semantic similarity in a taxonomy. In In Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-95, Morgan Kaufmann (1995), 448--453.
17. Rogers, Y. Icons at the interface: Their usefulness. vol. 1, Elsevier Science Inc. (New York, NY, USA, Apr. 1989), 105--117.
18. Rother, C., and Kolmogorov, V.and Blake, A. "grabcut": Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (TOG) 23, 3 (Aug. 2004), 309--314.
19. Setlur, V., Albrecht-Buehler, C., Gooch, A., Rossoff, S., and Gooch, B. Semanticons: Visual metaphors as file icons. vol. 24 (2005), 647--656.
20. Shneiderman, B. Designing the User Interface: Strategies for Effective Human-computer Interaction. Addison-Wesley Longman Publishing Company, Inc., Boston, MA, USA, 1986.
21. Smith, L., and Thomas, D. Color versus shape coding in information displays. vol. 48, Journal of Applied Psychology (1964), 137--146.
22. Wobbrock, J. O., Forlizzi, J., Hudson, S. E., and Myers, B. A. Webthumb: Interaction techniques for small-screen browsers. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology (UIST), ACM (New York, NY, USA, 2002), 205--208.
23. Woodruff, A., Faulring, A., Rosenholtz, R., Morrsion, J., and Pirolli, P. Using thumbnails to search the web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI), ACM (New York, NY, USA, 2001), 198--205.
24. Wu, Z., and Palmer, M. Verbs semantics and lexical selection. In Proceedings of the 32nd Annual Meeting on Association for Computational Linguistics, ACL '94, Association for Computational Linguistics (Stroudsburg, PA, USA, 1994), 133--138.

Index Terms

• Human-centered computing