Skip to main content
SpringerLink
Log in

Visual Blending for Concept Representation: A Case Study on Emoji Generation

  • Published:
New Generation Computing Aims and scope Submit manuscript

Abstract

The emoji connection between visual representation and semantic knowledge, together with its large conceptual coverage have the potential to be exploited in computational approaches to the visual representation of concepts. An example of a system that explores this potential is Emojinating—a system that uses a process of visual blending of existing emoji to represent concepts. In this paper, we use the Emojinating system as a case study to analyse the appropriateness of visual blending for the visual representation of concepts. We conduct three experiments in which we analyse output quality, type of blend used, usefulness to the user and ease of interpretation. Our main contributions are the following: (i) the production of a double-word concept list for testing the system; (ii) an extensive user study using two different concept lists (single-word and double-word); and (iii) a study that compares produced blends with user drawings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Abbing, R.R., Pierrot, P., Snelting, F.: Modifying the universal. Executing Practices p. 33 (2017)

  2. Adobe: Emoji trend report 2019. https://www.slideshare.net/adobe/adobe-emoji-trend-report-2019/1 (2019). [Online; accessed May 2020]

  3. Ai, W., Lu, X., Liu, X., Wang, N., Huang, G., Mei, Q.: Untangling emoji popularity through semantic embeddings. In: ICWSM, pp. 2–11 (2017)

  4. Barbieri, F., Ronzano, F., Saggion, H.: What does this emoji mean? a vector space skip-gram model for twitter emojis. In: LREC (2016)

  5. BRIS: Bris: Abused emojis. http://webcollection.se/bris/abusedemojis/ (2015). Accessed: January 2020

  6. Browne, C.: A new general service list: The better mousetrap we’ve been looking for. Vocabulary Learning and Instruction 3(1), 1–10 (2014)

    Google Scholar 

  7. Brysbaert, M., Warriner, A.B., Kuperman, V.: Concreteness ratings for 40 thousand generally known english word lemmas. Behav. Res. Methods 46(3), 904–911 (2014)

    Article  Google Scholar 

  8. Chilton, L.B., Petridis, S., Agrawala, M.: Visiblends: A flexible workflow for visual blends. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Glasgow, Scotland, UK, May 04-09, 2019, p. 172 (2019). https://doi.org/10.1145/3290605.3300402

  9. Confalonieri, R., Corneli, J., Pease, A., Plaza, E., Schorlemmer, M.: Using argumentation to evaluate concept blends in combinatorial creativity. In: Proceedings of the Sixth International Conference on Computational Creativity, pp. 174–181 (2015)

  10. Cunha, J.M., Gonçalves, J., Martins, P., Machado, P., Cardoso, A.: A pig, an angel and a cactus walk into a blender: a descriptive approach to visual blending. In: Proceedings of the Eighth International Conference on Computational Creativity (2017)

  11. Cunha, J.M., Lourenço, N., Correia, J., Martins, P., Machado, P.: Emojinating: evolving emoji blends. In: Proceedings of the Eighth International Conference on Computational Intelligence in Music, Sound, Art and Design, pp. 110–126. Springer (2019)

  12. Cunha, J.M., Lourenço, N., Martins, P., Machado, P.: Emojinating co-creativity: integrating self-evaluation and context-adaptation. In: Proceedings of the Eleventh International Conference on Computational Creativity (2020)

  13. Cunha, J.M., Martins, P., Machado, P.: Emojinating: representing concepts using emoji. In: Workshop Proceedings of the Twenty-Sixth International Conference on Case-Based Reasoning (2018)

  14. Cunha, J.M., Martins, P., Machado, P.: How shell and horn make a unicorn: experimenting with visual blending in emoji. In: Proceedings of the Ninth International Conference on Computational Creativity, Salamanca, Spain, June 25-29, 2018. (2018)

  15. Cunha, J.M., Martins, P., Machado, P.: Let’s figure this out: a roadmap for visual conceptual blending. In: Proceedings of the Eleventh International Conference on Computational Creativity (2020)

  16. Cunha, J.M., Rebelo, S., Martins, P., Machado, P.: Assessing usefulness of a visual blending system: “Pictionary has used image-making new meaning logic for decades. We Don’t Need a Computational Platform to Explore the Blending Phenomena”, Do We? In: Proceedings of the Tenth International Conference on Computational Creativity, UNC Charlotte, North Carolina, June 17-21, 2019. (2019)

  17. Dimson, T.: Emojineering part 1: machine learning for emoji trends (2015)

  18. Donato, G., Paggio, P.: Investigating redundancy in emoji use: study on a twitter based corpus. In: Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, pp. 118–126 (2017)

  19. Dürscheid, C., Siever, C.M.: Beyond the alphabet–communcataion of emojis. Kurzfassung eines (auf Deutsch) zur Publikation eingereichten Manuskripts (2017)

  20. Eisner, B., Rocktäschel, T., Augenstein, I., Bosnjak, M., Riedel, S.: emoji2vec: learning emoji representations from their description. In: Proceedings of The Fourth International Workshop on Natural Language Processing for Social Media (2016)

  21. Fares, M.: A dataset for joint noun–noun compound bracketing and interpretation. In: Proceedings of the ACL 2016 Student Research Workshop, pp. 72–79. Association for Computational Linguistics (2016)

  22. Fauconnier, G., Turner, M.: The Way We Think. Basic Books, New York (2002)

    Google Scholar 

  23. Finland promotion board: finland emojis. http://finland.fi/emoji/ (2016). Accessed: January 2020

  24. Ha, D., Eck, D.: A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2017)

  25. Ho, V.: Bloody brilliant: new emoji to symbolize menstruation welcomed. https://www.theguardian.com/technology/2019/feb/09/period-emoji-menstruation-blood-donation (2019). Accessed: July 2020

  26. Karimi, P., Maher, M.L., Grace, K., Davis, N.: A computational model for visual conceptual blends. IBM J. Res. Dev. (2018)

  27. Liapis, A., Yannakakis, G.N., Alexopoulos, C., Lopes, P.: Can computers foster human users’ creativity? theory and praxis of mixed-initiative co-creativity. Dig. Culture Educ. 8(2), 136–153 (2016)

    Google Scholar 

  28. Miller, H., Thebault-Spieker, J., Chang, S., Johnson, I., Terveen, L., Hecht, B.: Blissfully happy” or “ready to fight”: Varying interpretations of emoji. Proceedings of ICWSM 2016, (2016)

  29. Neff, J.: Dove launches curly-haired emojis to end straight-hair dominance: brand rescues wavy haired ’marginalized’ by emojis. https://adage.com/article/digital/dove-launches-curly-haired-emojis-address-void/301203 (2015). Accessed: January 2020

  30. Novak, P.K., Smailović, J., Sluban, B., Mozetič, I.: Sentiment of emojis. PloS One 10(12), e0144296 (2015)

    Article  Google Scholar 

  31. Padró, L., Stanilovsky, E.: Freeling 3.0: towards wider multilinguality. In: LREC2012 (2012)

  32. Pereira, F.C., Cardoso, A.: The boat-house visual blending experience. In: Proceedings of the Symposium for Creativity in Arts and Science of AISB 2002 (2002)

  33. Phillips, B.J., McQuarrie, E.F.: Beyond visual metaphor: a new typology of visual rhetoric in advertising. Marketing theory 4(1–2), 113–136 (2004)

    Article  Google Scholar 

  34. Prada, M., Rodrigues, D., Silva, R.R., Garrido, M.V.: Lisbon symbol database (lsd): subjective norms for 600 symbols. Behav. Res. Methods 48(4), 1370–1382 (2016)

    Article  Google Scholar 

  35. Puyat, M.: Emotigan: Emoji art using generative adversarial networks (2017). CS229: Machine Learning Course, Stanford University

  36. Radpour, D., Bheda, V.: Conditional generative adversarial networks for emoji synthesis with word embedding manipulation. arXiv preprint arXiv:1712.04421 (2017)

  37. Roberts, W., Egg, M.: A large automatically-acquired all-words list of multiword expressions scored for compositionality. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) (2018)

  38. Speer, R., Havasi, C.: Representing general relational knowledge in conceptnet 5. In: Proceedings of the Eighth International Conference on Language Resources and Evaluation, pp. 3679–3686 (2012)

  39. Twitter: Twitter emoji (twemoji). https://github.com/twitter/twemoji. Accessed: January 2020

  40. Venkatesh, V., Brown, S.A., Bala, H.: Bridging the qualitative-quantitative divide: Guidelines for conducting mixed methods research in information systems. MIS quarterly 37(1), (2013)

  41. Wicke, P.: Ideograms as semantic primes: Emoji in computational linguistic creativity (2017)

  42. Wicke, P., Bolognesi, M.: Emoji-based semantic representations for abstract and concrete concepts. Cognitive processing (2020)

  43. Wicke, P., Cunha, J.M.: :books: :arrow\_right: :slight\_smile: an approach for text-to-emoji. In: Proceedings of the Eleventh International Conference on Computational Creativity (2020)

  44. Wijeratne, S., Balasuriya, L., Sheth, A., Doran, D.: Emojinet: an open service and api for emoji sense discovery. In: Proceedings of ICWSM-17 (2017)

  45. Xiao, P., Linkola, S.: Vismantic: Meaning-making with images. In: Proceedings of the Sixth International Conference on Computational Creativity, ICCC-15 (2015)

  46. Zhao, N., Kim, N.W., Herman, L.M., Pfister, H., Lau, R.W., Echevarria, J., Bylinskii, Z.: Iconate: automatic compound icon generation and ideation. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (2020)

Download references

Author information

Authors and Affiliations

  1. Department of Informatics Engineering, Centre for Informatics and Systems of the University of Coimbra, Coimbra, Portugal

    João M. Cunha, Nuno Lourenço, Pedro Martins & Penousal Machado

Authors
  1. João M. Cunha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nuno Lourenço
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pedro Martins
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Penousal Machado
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to João M. Cunha.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work is partially funded by national funds through the FCT - Foundation for Science and Technology, I.P., within the scope of the project CISUC - UID/CEC/00326/2020 and by European Social Fund, through the Regional Operational ProgramCentro 2020, and under the grant SFRH/BD/120905/2016. This work includes data from ConceptNet 5, which was compiled by the Commonsense Computing Initiative and is freely available under the Creative Commons Attribution-ShareAlike license (CC BY SA 4.0) from conceptnet.io.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cunha, J.M., Lourenço, N., Martins, P. et al. Visual Blending for Concept Representation: A Case Study on Emoji Generation. New Gener. Comput. 38, 739–771 (2020). https://doi.org/10.1007/s00354-020-00107-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00354-020-00107-x

Keywords

Search

Navigation