Skip to main content
SpringerLink
Log in
Book cover

European Conference on Information Retrieval

ECIR 2023: Advances in Information Retrieval pp 544–552Cite as

Towards Detecting Interesting Ideas Expressed in Text

  • Conference paper
  • First Online:

  • 1467 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13981))

Abstract

In recent years, product and project ideas are often sourced from public competitions, where anyone can enter their own solutions to an open-ended question. While copious ideas can be gathered in this way, it becomes difficult to find the most promising results among all entries. This paper explores the potential of automating the detection of interesting ideas and studies the effect of various features of ideas on the prediction task. A BERT-based model is built to rank ideas by their predicted interestingness, using text embeddings from idea descriptions and the concreteness, novelty as well as the uniqueness of ideas. The model is trained on a dataset of OpenIDEO idea competitions. The results show that language models can be used to speed up finding promising ideas, but care must be taken in choosing a suitable dataset.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://www.openideo.com/.

  2. 2.

    https://pytorch.org/.

  3. 3.

    https://huggingface.co/docs/transformers/model_doc/distilbert#transformers.DistilBertModel/.

  4. 4.

    https://www.openideo.com/challenges.

References

  1. Adamopoulos, P., Tuzhilin, A.: On unexpectedness in recommender systems: Or how to better expect the unexpected. ACM Trans. Intell. Syst. Technol. (TIST) 5(4), 1–32 (2014)

    Google Scholar 

  2. Ahmed, F., Fuge, M.: Capturing winning ideas in online design communities. In: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, ACM, New York, NY, USA (2017). https://doi.org/10.1145/2998181.2998249

  3. Ahmed, F., Fuge, M.: Ranking ideas for diversity and quality. J. Mech. Des. 140(1) (2018). https://doi.org/10.1115/1.4038070

  4. Blohm, I., Riedl, C., Füller, J., Leimeister, J.M.: Rate or trade? identifying winning ideas in open idea sourcing. Inf. Syst. Res. 27(1), 27–48 (2016). https://doi.org/10.1287/isre.2015.0605

    Article  Google Scholar 

  5. Breunig, M.M., Kriegel, H.P., Ng, R.T., Sander, J.: Lof. In: Dunham, M. (ed.) Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93–104. ACM Conferences, ACM, New York, NY (2000). https://doi.org/10.1145/342009.335388

  6. Fuge, M., Tee, K., Agogino, A., Maton, N.: Analysis of collaborative design networks: a case study of openideo. J. Comput. Inf. Sci. Eng. 14(2) (2014). https://doi.org/10.1115/1.4026510

  7. Jatowt, A., Hung, I.-C., Färber, M., Campos, R., Yoshikawa, M.: Exploding TV sets and disappointing laptops: suggesting interesting content in news archives based on surprise estimation. In: Hiemstra, D., Moens, M.-F., Mothe, J., Perego, R., Potthast, M., Sebastiani, F. (eds.) ECIR 2021. LNCS, vol. 12656, pp. 254–269. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72113-8_17

    Chapter  Google Scholar 

  8. Kuznetsov, S.O., Makhalova, T.: On interestingness measures of formal concepts. Inf. Sci. 442–443, 202–219 (2018). https://doi.org/10.1016/j.ins.2018.02.032

    Article  MathSciNet  MATH  Google Scholar 

  9. Loshchilov, I., Hutter, F.: Fixing weight decay regularization in adam. CoRR abs/1711.05101 (2017)

    Google Scholar 

  10. Naveed, N., Gottron, T., Kunegis, J., Alhadi, A.C.: Bad news travel fast: a content-based analysis of interestingness on twitter. In: Proceedings of the 3rd International Web Science Conference, pp. 1–7 (2011)

    Google Scholar 

  11. Paetzold, G., Specia, L.: Inferring psycholinguistic properties of words. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Stroudsburg, PA, USA (2016). https://doi.org/10.18653/v1/n16-1050

  12. Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using siamese BERT-networks. arXiv (2019). https://doi.org/10.48550/arXiv.1908.10084

  13. Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you?”: Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016, pp. 1135–1144 (2016)

    Google Scholar 

  14. Sanh, V., Debut, L., Chaumond, J., Wolf, T.: Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. CoRR abs/1910.01108 (2019)

    Google Scholar 

  15. Sun, Y., Wong, A.K.C., Kamel, M.S.: Classification of imbalanced data: a review. Int. J. Pattern Recogn. Artif. Intell. 23(04), 687–719 (2009)

    Article  Google Scholar 

  16. Terwiesch, C., Xu, Y.: Innovation contests, open innovation, and multiagent problem solving. Manage. Sci. 54(9), 1529–1543 (2008). https://doi.org/10.1287/mnsc.1080.0884

    Article  Google Scholar 

  17. Dasgupta, T., Dey, L.: Automatic scoring for innovativeness of textual ideas (2016)

    Google Scholar 

  18. van der Burgt, M.: Calibrating low-default portfolios, using the cumulative accuracy profile. J. Risk Model Validation 1(4), 17–33 (2008)

    Article  Google Scholar 

  19. Wahl, J., Füller, J., Hutter, K.: What’s the problem? how crowdsourcing and text-mining may contribute to the understanding of unprecedented problems such as covid-19. R &D Manage. 52(2), 427–446 (2022). https://doi.org/10.1111/radm.12526

    Article  Google Scholar 

  20. Baba, Y., Li, J., Kashima, H.: Crowdea: multi-view idea prioritization with crowds. In: Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, vol. 8, pp. 23–32 (2020). https://ojs.aaai.org/index.php/hcomp/article/view/7460

  21. Zhang, Y., Siriaraya, P., Kawai, Y., Jatowt, A.: Predicting time and location of future crimes with recommendation methods. Knowledge-Based Systems 210, 106503 (2020). https://doi.org/10.1016/j.knosys.2020.106503

Download references

Acknowledgment

The dataset consisting of 21 OpenIDEO competitions used in the paper has been provided by the research group for Innovation & Entrepreneurship (https://www.uibk.ac.at/smt/innovation-entrepreneurship/) at the Department of Strategic Management, Marketing and Tourism of the University of Innsbruck. Parts of the experiments have been conducted while Bela Pfahl was employed as a student research assistant in the group.

Author information

Authors and Affiliations

  1. University of Innsbruck, Innsbruck, Austria

    Bela Pfahl & Adam Jatowt

Authors
  1. Bela Pfahl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adam Jatowt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adam Jatowt .

Editor information

Editors and Affiliations

  1. University of Amsterdam, Amsterdam, The Netherlands

    Jaap Kamps

  2. Université Grenoble-Alpes, Saint-Martin-d’Hères, France

    Lorraine Goeuriot

  3. Università della Svizzera Italiana, Lugano, Switzerland

    Fabio Crestani

  4. University of Copenhagen, Copenhagen, Denmark

    Maria Maistro

  5. University of Tsukuba, Ibaraki, Japan

    Hideo Joho

  6. Dublin City University, Dublin, Ireland

    Brian Davis

  7. Dublin City University, Dublin, Ireland

    Cathal Gurrin

  8. Universität Regensburg, Regensburg, Germany

    Udo Kruschwitz

  9. Dublin City University, Dublin, Ireland

    Annalina Caputo

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pfahl, B., Jatowt, A. (2023). Towards Detecting Interesting Ideas Expressed in Text. In: Kamps, J., et al. Advances in Information Retrieval. ECIR 2023. Lecture Notes in Computer Science, vol 13981. Springer, Cham. https://doi.org/10.1007/978-3-031-28238-6_45

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-28238-6_45

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-28237-9

  • Online ISBN: 978-3-031-28238-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation