skip to main content
10.1145/3428658.3431083acmconferencesArticle/Chapter ViewAbstractPublication PageswebmediaConference Proceedingsconference-collections
research-article

A Crowdsourcing Method for Sign Segmentation in Brazilian Sign Language Videos

Published: 30 November 2020 Publication History

Abstract

Like the spoken languages, sign languages are not universal and vary in different countries. LIBRAS (Brazilian Sign Language) is the second official language of Brazil and it is the language adopted by Brazilian Deaf's community to communicate. The signs of LIBRAS are composed of hand configurations, facial expressions and are affected by space and intensity modifiers, which makes their recognition more complicated than the simple identification of hand signs. The signs are arranged, according to a grammar, respecting form phrases, clauses, and sentences like any other spoken or sign language. The automatic machine translation of a sign language typically includes an initial phase for detecting sign boundaries. In this paper, we apply a crowdsourcing method to identifying signs boundaries present in pre-recorded videos those features LIBRAS interpreters. The limits or boundaries of the signs in the videos were established from the processing of contributions from workers from different countries, who have supposedly never heard of LIBRAS nor any other sign languages. To evaluate the segmentation process, we compared the sign boundaries identified by the crowd with the ground truth provided by a team of LIBRAS experts, who also assessed the quality of the delimitation of the identified signs. Our analysis showed that our crowdsourcing method was able to get 93.75% of the sign boundaries successfully.

References

[1]
M. N. Amorim, F. R. Assis Neto, and C. A. S. Santos. 2018. Achieving Complex Media Annotation through Collective Wisdom and Effort from the Crowd. In 2018 25th International Conference on Systems, Signals and Image Processing (IWSSIP). IEEE, 1--5. https://doi.org/10.1109/IWSSIP.2018.8439402
[2]
F. R. Assis Neto and C. A. S. Santos. 2018. Understanding crowdsourcing projects: A systematic review of tendencies, workflow, and quality management. Information Processing & Management 54, 4 (2018), 490--506. https://doi.org/10.1016/j.ipm.2018.03.006
[3]
M. Chavent, F. de A.T. de Carvalho, Y. Lechevallier, and R. Verde. 2006. New clustering methods for interval data. Computational statistics 21, 2 (2006), 211--229. https://doi.org/10.lO07/sO0180-006-0260-0
[4]
M.N. de Amorim, R.M.C. Costa Segundo, C.A.S. Santos, and O.L. Tavares. 2017. Video Annotation by Cascading Microtasks: A Crowdsourcing Approach. In 23rd Brazillian Symposium on Multimedia and the Web (Gramado, RS, Brazil) (WebMedia '17). Association for Computing Machinery, New York, NY, USA, 49--56. https://doi.org/10.1145/3126858.3126897
[5]
M. N. de Amorim, E. B. Saleme, F. R. Assis Neto, C. A. S. Santos, and G. Ghinea. 2019. Crowdsourcing authoring of sensory effects on videos. Multimedia Tools and Applications (08 Feb 2019). https://doi.org/10.1007/s11042-019-7312-2
[6]
I. Hernández. 2018. Automatic Irish Sign Language Recognition. Ph.D. Dissertation. University of Dublin, Trinity College. https://scss.tcd.ie/publications/theses/diss/2018/TCD-SCSS-DISSERTATION-2018-034.pdf Doctoral.
[7]
Hong Hong, Qianzhou Du, Gang Wang, Weiguo Fan, and Di Xu. 2016. Crowd Wisdom: The Impact of Opinion Diversity and Participant Independence on Crowd Performance. In AMCIS.
[8]
J. Huang, W. Zhou, Q. Zhang, H. Li, and W. Li. 2018. Video-based sign language recognition without temporal segmentation. In Thirty-Second AAAI Conference on Artificial Intelligence.
[9]
N.Q.V. Hung, N.T. Tam, L.N. Tran, and K. Aberer. 2013. An evaluation of aggregation techniques in crowdsourcing. In International Conference on Web Information Systems Engineering. Springer, 1--15.
[10]
S. Khan. 2014. Segmentation of continuous sign language. Ph.D. Dissertation. Massey University. http://hdl.handle.net/10179/6945 Doctoral.
[11]
K. Lee, J. Caverlee, and S. Webb. 2010. The social honeypot project: protecting online communities from spammers. In Proc. of the 19th International Conference on Worldwide web. ACM, 1139--1140.
[12]
S. Liwicki and M. Everingham. 2009. Automatic recognition of fingerspelled words in British Sign Language. In 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. 50--57. https://doi.org/10.1109/CVPRW.2009.5204291
[13]
M. Matsangidou, J. Otterbacher, C. S. Ang, and P. Zaphiris. 2018. Can the crowd tell how I feel? Trait empathy and ethnic background in a visual pain judgment task. Universal Access in the Information Society 17, 3 (01 Aug 2018), 649--661. https://doi.org/10.1007/s10209-018-0611-y
[14]
A.J. Porfirio, K.L. Wiggers, L.E.S. Oliveira, and D. Weingaertner. 2013. LIBRAS Sign Language Hand Configuration Recognition Based on 3D Meshes. In 2013 IEEE International Conference on Systems, Man, and Cybernetics (SMC '13). 1588--1593. https://doi.org/10.1109/SMC.2013.274
[15]
L. Von Ahn. 2005. Human Computation. Ph.D. Dissertation. Carnegie Mellon University, Pittsburgh, PA, USA. Advisor(s) Blum, Manuel. AAI3205378.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
WebMedia '20: Proceedings of the Brazilian Symposium on Multimedia and the Web
November 2020
364 pages
ISBN:9781450381963
DOI:10.1145/3428658
© 2020 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

In-Cooperation

  • SBC: Brazilian Computer Society
  • CNPq: Conselho Nacional de Desenvolvimento Cientifico e Tecn
  • CGIBR: Comite Gestor da Internet no Brazil
  • CAPES: Brazilian Higher Education Funding Council

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 November 2020

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Crowdsourcing
  2. LIBRAS
  3. segmentation
  4. translation video

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • Coordenação de Aperfeiçoamento de Pessoal de Nível Superior

Conference

WebMedia '20
Sponsor:
WebMedia '20: Brazillian Symposium on Multimedia and the Web
November 30 - December 4, 2020
São Luís, Brazil

Acceptance Rates

WebMedia '20 Paper Acceptance Rate 34 of 87 submissions, 39%;
Overall Acceptance Rate 270 of 873 submissions, 31%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 64
    Total Downloads
  • Downloads (Last 12 months)7
  • Downloads (Last 6 weeks)2
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media