Henning Pohl
Christian Domin
Michael Rohs
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Emoji, a set of pictographic Unicode characters, have seen strong uptake over the last couple of years. All common mobile platforms and many desktop systems now support emoji entry, and users have embraced their use. Yet, we currently know very little about what makes for good emoji entry. While soft keyboards for text entry are well optimized, based on language and touch models, no such information exists to guide the design of emoji keyboards. In this article, we investigate of the problem of emoji entry, starting with a study of the current state of the emoji keyboard implementation in Android. To enable moving forward to novel emoji keyboard designs, we then explore a model for emoji similarity that is able to inform such designs. This semantic model is based on data from 21 million collected tweets containing emoji. We compare this model against a solely description-based model of emoji in a crowdsourced study. Our model shows good perfor mance in capturing detailed relationships between emoji.
Supplemental Material
Available for Download
zip
pohl.zip (82.7 MB)
Supplemental file
- Sho Aoki and Osamu Uchida. 2011. A method for automatically generating the emotional vectors of emoticons using weblog articles. In Proceedings of the 10th WSEAS International Conference on Applied Computer and Applied Computational Science (ACACOS’11). 132--136.Google Scholar
Digital Library
- Gilles Bailly, Antti Oulasvirta, Duncan P. Brumby, and Andrew Howes. 2014. Model of visual search and selection time in linear menus. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI’14). ACM, New York, NY, 3865--3874. DOI:http://dx.doi.org/10.1145/2556288.2557093 Google ScholarDigital Library
- Saeideh Bakhshi, David A. Shamma, Lyndon Kennedy, Yale Song, Paloma de Juan, and Joseph ‘Jofish’ Kaye. 2016. Fast, cheap, and good: Why animated GIFs engage us. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI’16). ACM Press, New York, NY, 575--586. DOI:http://dx.doi.org/10.1145/2858036.2858532 Google ScholarDigital Library
- Xiaojun Bi, Barton A. Smith, and Shumin Zhai. 2012. Multilingual touchscreen keyboard design and optimization. Human Computer Interaction 27, 4 (2012), 352--382. DOI:http://dx.doi.org/10.1080/07370024.2012.678241Google Scholar
- Matthias Böhmer, Christian Lander, Sven Gehring, Duncan P. Brumby, and Antonio Krüger. 2014. Interrupted by a phone call: Exploring designs for lowering the impact of call notifications for smartphone users. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI’14). ACM Press, New York, NY, 3045--3054. DOI:http://dx.doi.org/10.1145/2556288.2557066 Google ScholarDigital Library
- Spencer Cappallo, Thomas Mensink, and Cees G. M. Snoek. 2015a. Image2Emoji: Zero-shot emoji prediction for visual media. In Proceedings of the 23rd ACM International Conference on Multimedia. 1311--1314. DOI:http://dx.doi.org/10.1145/2733373.2806335 Google ScholarDigital Library
- Spencer Cappallo, Thomas Mensink, and Cees G. M. Snoek. 2015b. Query-by-Emoji video search. In Proceedings of the 23rd ACM International Conference on Multimedia. 735--736. DOI:http://dx.doi.org/10.1145/2733373.2807961 Google ScholarDigital Library
- Heeryon Cho and Toru Ishida. 2011. Exploring cultural differences in pictogram interpretations. In The Language Grid, Toru Ishida (Ed.). Springer, Berlin, Heidelberg, 133--148. DOI:http://dx.doi.org/10.1007/978-3-642-21178-2_9 Google ScholarCross Ref
- Karen Church, Denzil Ferreira, Nikola Banovic, and Kent Lyons. 2015. Understanding the challenges of mobile phone usage data. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI’15). ACM Press, New York, NY, 504--514. DOI:http://dx.doi.org/10.1145/2785830.2785891 Google ScholarDigital Library
- Mark Dunlop and John Levine. 2012. Multidimensional Pareto optimization of touchscreen keyboards for speed, familiarity and improved spell checking. In Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems (CHI’12). ACM Press, New York, NY, 2669--2678. DOI:http://dx.doi.org/10.1145/2207676.2208659 Google ScholarDigital Library
- Thomas B. Fitzpatrick. 1988. The validity and practicality of sun-reactive skin types I through VI. Archives of Dermatology 124, 6 (1988), 869--871. DOI:http://dx.doi.org/10.1001/archderm.1988.01670060015008 Google ScholarCross Ref
- Jeffrey T. Hancock, Christopher Landrigan, and Courtney Silver. 2007. Expressing emotion in text-based communication. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’07). ACM Press, New York, NY, 929--932. DOI:http://dx.doi.org/10.1145/1240624.1240764 Google ScholarDigital Library
- Niels Henze, Enrico Rukzio, and Susanne Boll. 2012. Observational and experimental investigation of typing behaviour using virtual keyboards for mobile devices. In Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems (CHI’12). ACM, 2659--2668. DOI:http://dx.doi.org/10.1145/2208636.2208658Google ScholarDigital Library
- Joris H. Janssen, Wijnand A. Ijsselsteijn, and Joyce H. D. M. Westerink. 2014. How affective technologies can influence intimate interactions and improve social connectedness. International Journal of Human Computer Studies 72, 1 (2014), 33--43. DOI:http://dx.doi.org/10.1016/j.ijhcs.2013.09.007 Google ScholarDigital Library
- Andreas Karrenbauer and Antti Oulasvirta. 2014. Improvements to keyboard optimization with integer programming. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST’14). ACM Press, New York, NY, 621--626. DOI:http://dx.doi.org/10.1145/2642918.2647382 Google ScholarDigital Library
- David E. Kieras and Anthony J. Hornof. 2014. Towards accurate and practical predictive models of active-vision-based visual search. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI’14). ACM, New York, NY, 3875--3884. DOI:http://dx.doi.org/10.1145/2556288.2557324 Google ScholarDigital Library
- Per Ola Kristensson and Keith Vertanen. 2014. The inviscid text entry rate and its application as a grand goal for mobile text entry. In Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices 8 Services (MobileHCI’14). ACM Press, New York, NY, 335--338. DOI:http://dx.doi.org/10.1145/2628363.2628405 Google ScholarDigital Library
- Henry Kučera and W. Nelson Francis. 1967. Computational Analysis of Present-Day American English. Brown University Press, Providence, RI.Google Scholar
- Lisa Lebduska. 2015. Emoji, Emoji, What for art thou? Harlot 12 (2015). http://harlotofthearts.org/index.php/harlot/article/view/186/157.Google Scholar
- Ying Liu and Qiqun Wang. 2007. Chinese pinyin phrasal input on mobile phone: Usability and developing trends. In Proceedings of the 4th International Conference on Mobile Technology, Applications, and Systems and the 1st International Symposium on Computer Human Interaction in Mobile Technology (Mobility’07), Vol. 07. ACM Press, New York, NY, 540--546. DOI:http://dx.doi.org/10.1145/1378063.1378151 Google ScholarDigital Library
- Daniel P. Lopresti and Andrew Tomkins. 1993. Approximate matching of hand-drawn pictograms. In Proceedings of the Third International Workshop on Frontiers in Handwriting Recognition. 102--111.Google Scholar
- Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proceedings of NIPS.Google ScholarDigital Library
- Hannah Miller, Jacob Thebault-Spieker, Shuo Chang, Isaac Johnson, Loren Terveen, and Brent Hecht. 2016. Blissfully happy or ready to fight: Varying interpretations of emoji. In Proceedings of International AAAI Conference on Web and Social Media. 259--268.Google Scholar
- Marketta Niemelä and Jukka Saarinen. 2000. Visual search for grouped versus ungrouped icons in a computer interface. Human Factors: The Journal of the Human Factors and Ergonomics Society 42, 4 (2000), 630--635. DOI:http://dx.doi.org/10.1518/001872000779697999 Google ScholarCross Ref
- Petra Kralj Novak, Jasmina Smailovic, Borut Sluban, and Igor Mozetic. 2015. Sentiment of emojis. PLoS ONE 10, 12 (2015), 1--19. DOI:http://dx.doi.org/10.1371/journal.pone.0144296 Google ScholarCross Ref
- Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesna. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825--2830.Google ScholarDigital Library
- Henning Pohl and Roderick Murray-Smith. 2013. Focused and casual interactions: Allowing users to vary their level of engagement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’13). ACM, New York, NY, 2223--2232. DOI:http://dx.doi.org/10.1145/2470654.2481307 Google ScholarDigital Library
- Henning Pohl, Dennis Stanke, and Michael Rohs. 2016. EmojiZoom: Emoji entry via large overview maps. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Companion (MobileHCI’16). DOI:http://dx.doi.org/10.1145/2935334.2935382 Google ScholarDigital Library
- Radim Řehůřek and Petr Sojka. 2010. Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. ELRA, Valetta, Malta, 45--50.Google Scholar
- Jared Suttles and Nancy Ide. 2013. Distant supervision for emotion classification with discrete binary values. In Proceedings of the 14th International Conference on Computational Linguistics and Intelligent Text Processing (CICLing’13), Vol. 7817 LNCS. 121--136. DOI:http://dx.doi.org/10.1007/978-3-642-37256-8_11 Google ScholarDigital Library
- C. C. Tappert, C. Y. Suen, and T. Wakahara. 1990. The state of the art in online handwriting recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 8 (1990), 787--808. DOI:http://dx.doi.org/10.1109/34.57669 Google ScholarDigital Library
- Chad C. Tossell, Philip Kortum, Clayton Shepard, Laura H. Barg-Walkow, Ahmad Rahmati, and Lin Zhong. 2012. A longitudinal study of emoticon use in text messaging from smartphones. Computers in Human Behavior 28, 2 (2012), 659--663. DOI:http://dx.doi.org/10.1016/j.chb.2011.11.012 Google ScholarDigital Library
- Yuki Urabe, Rafal Rzepka, and Kenji Araki. 2013. Emoticon recommendation system for effective communication. In Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM’13). ACM, New York, NY, 1460--1461. DOI:http://dx.doi.org/10.1145/2492517.2492594 Google ScholarDigital Library
- Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9 (2008), 2579--2605. DOI:http://dx.doi.org/10.1007/s10479-011-0841-3 Google ScholarCross Ref
- Leticia Vidal, Gastón Ares, and Sara R. Jaeger. 2015. Use of emoticon and emoji in tweets for food-related emotional expression. Food Quality and Preference 49 (2015), 119--128. DOI:http://dx.doi.org/10.1016/j.foodqual.2015.12.002 Google ScholarCross Ref
- Daryl Weir, Henning Pohl, Simon Rogers, Keith Vertanen, and Per Ola Kristensson. 2014. uncertain text entry on mobile devices. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI’14). ACM, New York, NY, 2307--2316. DOI:http://dx.doi.org/10.1145/2556288.2557412 Google ScholarDigital Library
- Karl D. D. Willis and Ivan Poupyrev. 2010. 12Pixels: Exploring social drawing on mobile phones. In Proceedings of the 8th International Conference on Pervasive Computing (Pervasive’10). Springer, Berlin, 391--408. DOI:http://dx.doi.org/10.1007/978-3-642-12654-3_23 Google ScholarDigital Library
- Shumin Zhai, Michael Hunter, and Barton A. Smith. 2000. The metropolis keyboard—An exploration of quantitative techniques for virtual keyboard design. In Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology (UIST’00). ACM, New York, NY, 119--128. DOI:http://dx.doi.org/10.1145/354401.354424 Google ScholarDigital Library
Index Terms
- Beyond Just Text: Semantic Emoji Similarity Modeling to Support Expressive Communication 👫📲😃
Recommendations
The Interplay Between Emojis and Linguistic Text
Chinese Lexical SemanticsAbstractThis paper examines the interplay between emojis and the accompanying linguistic text when the two channels disagree with each other. In general, emojis are typically regarded as an explicit way to convey a specific emotion. However, it is ...
Perceived responsiveness in text messaging: The role of emoji use
AbstractThis research investigated how emojis can be used in text messaging to communicate perceived responsiveness, guide impression formation, and contribute to reflected appraisal. Participants (N = 179) disclosed a positive and negative ...
Highlights- Combination of discloser and responder emoji use affects person perceptions.
- ...
The communicative role of non-face emojis
Emojis have evolved from imitations of facial expressions meant to communicate affect into pictures of objects, food, and places that are not directly linked to affect. While emojis that resemble facial expressions are well-researched, emojis that ...
Comments