Skip to main content
SpringerLink
Log in

Crowdsourced object-labeling based on a game-based mobile application

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Unparalleled growth in the sharing of media via networks has prompted a great deal of research into issues pertaining to image retrieval. The training and verification of image retrieval systems requires a large number of labelled images with ground truth; however, most researchers employ public datasets for their experiments, the results are restricted by the size and content of the dataset. In this study, we developed a system based on a mobile phone App for the collection of information pertaining to the location of objects in images. The proposed system is simple and easy to use. Experiments demonstrate the excellent performance of the proposed system with regard to accuracy and response time. This study demonstrates the feasibility of collecting image information using mobile phones.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26

Similar content being viewed by others

References

  1. Bannour H, Hudelot C (2012) Hierarchical image annotation using semantic hierarchies. In: Proceedings of the 21st ACM international conference on information and knowledge management. ACM, pp 2431–2434

  2. Captcha forms – good or evil? (2012). https://www.hallaminternet.com/captcha-forms-good-or-evil/

  3. Chellapilla K, Larson K, Simard P, Czerwinski M (2005) Designing human friendly human interaction proofs (hips). In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 711–720

  4. Chen KH, Wu CR, Yang Y, Huang JW, Ho TY (2014) Efficient building identification using structural and spatial information on mobile devices. In: 2014 IEEE international conference on multimedia and expo workshops (ICMEW). IEEE, pp 1–5

  5. Customizing the look and feel of recaptcha (2016). https://developers.google.com/recaptcha/old/docs/customization

  6. Datta R, Joshi D, Li J, Wang J (2008) Image retrieval: ideas, influences, and trends of the new age. ACM Comput Surv (CSUR) 40(2):5

    Article  Google Scholar 

  7. Eom W, Lee S, De Neve W, Ro YM (2011) Improving image tag recommendation using favorite image context. In: 2011 18th IEEE international conference on image processing. IEEE, pp 2445–2448

  8. Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html

  9. Gao Y, Peng J, Luo H, Keim DA, Fan J (2009) An interactive approach for filtering out junk images from keyword-based google search results. IEEE Trans Circ Syst Video Technol 19(12):1851–1865

    Article  Google Scholar 

  10. Google now using recaptcha to decode street view addresses (2012). https://cola.workxplay.net/what-is-google-recaptcha/

  11. Huang L, Yang Y, Deng Y, Yu Y (2015) Densebox: unifying landmark localization with end to end object detection. arXiv:1509.04874

  12. Jing P, Su Y, Nie L, Gu H, Liu J, Wang M (2018) A framework of joint low-rank and sparse regression for image memorability prediction. IEEE Transactions on Circuits and Systems for Video Technology, pp 1–1. https://doi.org/10.1109/TCSVT.2018.2832095

  13. Joachims T (2002) Optimizing search engines using clickthrough data. In: Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 133–142

  14. Lerthirunwong N, Shimizu I (2012) Interactive metric learning system for similar image search using linear discriminant analysis. In: The 1st IEEE global conference on consumer electronics 2012. IEEE, pp 206–209

  15. Li B, Xiong W, Hu W (2011) Web horror image recognition based on context-aware multi-instance learning. In: 2011 IEEE 11th international conference on data mining. IEEE, pp 1158–1163

  16. Liu Y, Mei T (2011) Optimizing visual search reranking via pairwise learning. IEEE Trans Multimed 13(2):280–291

    Article  Google Scholar 

  17. Liu Y, Mei T, Hua XS (2009) Crowdreranking: exploring multiple search engines for visual search reranking. In: Proceedings of the 32nd international ACM SIGIR conference on research and development in information retrieval. ACM, pp 500–507

  18. Liu D, Hua XS, Wang M, Zhang HJ (2010) Image retagging. In: Proceedings of the 18th ACM international conference on multimedia. ACM, pp 491–500

  19. Liu Y, Wu F, Zhang Y, Shao J, Zhuang Y (2011) Tag clustering and refinement on semantic unity graph. In: 2011 IEEE 11th international conference on data mining. IEEE, pp 417–426

  20. Liu X, Qian X, Lu D, Hou X, Wang L (2014) Personalized tag recommendation for flickr users. In: 2014 IEEE International conference on multimedia and expo (ICME). IEEE, pp 1–6

  21. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (2007) Object retrieval with large vocabularies and fast spatial matching. In: 2007 IEEE conference on computer vision and pattern recognition. IEEE, pp 1–8

  22. Rae A, Sigurbjörnsson B, van Zwol R (2010) Improving tag recommendation using social networks. In: Adaptivity, personalization and fusion of heterogeneous information. Le Centre De Hautes Etudes Internationales D’Informatique Documentaire, pp 92–99

  23. recaptcha – i’m not a robot (2017). http://blog.rubyeffect.com/recaptcha-im-not-robot/

  24. Russell BC, Torralba A, Murphy KP, Freeman WT (2008) Labelme: a database and web-based tool for image annotation. Int J Comput Vis 77(1-3):157–173

    Article  Google Scholar 

  25. Su JH, Chou CL, Lin CY, Tseng VS (2011) Effective semantic annotation by image-to-concept distribution model. IEEE Trans Multimed 13(3):530–538

    Article  Google Scholar 

  26. Tang X, Liu K, Cui J, Wen F, Wang X (2012) Intentsearch: capturing user intention for one-click internet image search. IEEE Trans Pattern Anal Mach Intell 34(7):1342–1353

    Article  Google Scholar 

  27. Tian X, Tao D, Hua XS, Wu X (2010) Active reranking for web image search. IEEE Trans Image Process 19(3):805–820

    Article  MathSciNet  MATH  Google Scholar 

  28. Tian X, Yang L, Wang J, Wu X, Hua XS (2011) Bayesian visual reranking. IEEE Trans Multimed 13(4):639–652

    Article  Google Scholar 

  29. The clipper library performs clipping, and offsetting of both lines and polygons (2014). http://www.angusj.com/delphi/clipper/documentation/Docs/Overview/_Body.htm

  30. Uijlings JR, van de Sande KE, Gevers T, Smeulders AW (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171

  31. Von Ahn L, Blum M, Hopper NJ, Langford J (2003) Captcha: Using hard ai problems for security. In: International conference on the theory and applications of cryptographic techniques. Springer, pp 294–311

  32. Von Ahn L, Maurer B, McMillen C, Abraham D, Blum M (2008) recaptcha: human-based character recognition via web security measures. Science 321 (5895):1465–1468

    Article  MathSciNet  MATH  Google Scholar 

  33. Wan S, Jin P, Yue L (2011) An effective image retrieval technique based on color perception. In: 2011 sixth international conference on image and graphics (ICIG). IEEE, pp 1017–1022

  34. Yang L, Hanjalic A (2012) Prototype-based image search reranking. IEEE Trans Multimed 14(3):871–882

    Article  Google Scholar 

  35. Yang L, Hanjalic A (2013) Learning to rerank web images. IEEE MultiMedia 20(2):13–21

  36. Yao T, Ngo CW, Mei T (2013) Circular reranking for visual search. IEEE Trans Image Process 22(4):1644–1655

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, National Cheng Kung University, Tainan, Taiwan

    Wen-Yen Tseng & Jen-Wei Huang

  2. Graduate Program of Multimedia Systems and Intelligent Computing, National Cheng Kung University, Tainan, Taiwan

    Kai-Hsiang Chen

  3. Graduate Program of Multimedia Systems and Intelligent Computing, Academia Sinica, Taipei, Taiwan

    Kai-Hsiang Chen

Authors
  1. Wen-Yen Tseng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kai-Hsiang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jen-Wei Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kai-Hsiang Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tseng, WY., Chen, KH. & Huang, JW. Crowdsourced object-labeling based on a game-based mobile application. Multimed Tools Appl 78, 18137–18168 (2019). https://doi.org/10.1007/s11042-018-6944-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6944-y

Keywords

Search

Navigation