Exploring the Benefits of Text and Sketch in Video Retrieval of Complex Queries
Authors: 
Yan Huang, Cuixia Ma, Hongan WangAuthors Info & Claims
Yan Huang, Cuixia Ma, Hongan WangAuthors Info & ClaimsPages 115 - 120
Abstract
The booming of mobile devices and networks leads to an explosive growth in video resources. Efficient video research styles are appealing for facilely exploring video content appropriate to user's intention with a low cognitive load. The style of input becomes particularly relevant during the process of finding a target video clip in a large-scale database on tablets or other mobile devices. Some users have strong allegiance to input text, while others only input sketches. In this paper, we present the first systematic comparison of these two input styles and analyze the responses and feedbacks of users. An elaborated user study was conducted to test two different styles of inputting the semantics. Some users preferred to text input because it could describe their objective easily in a short time, yet some users also liked sketch because it helped illustrate the action or orientation clearly and immediately. Combining text with sketch ("sketch-text") is efficient for searching video of complex queries. The evaluation results show users' enjoying "sketch-text" and its higher performance than other input styles.
References
[1]
Google Video Search, http://video.google.com/.
[2]
Microsoft Bing Video Search, http://www.bing.com/videos.
[3]
YouTube Videos, https://www.youtube.com/videos.
[4]
Adcock, J., Cooper, M., Pickens, J. Experiments in interactive video search by addition and subtraction. Proceedings of the 2008 international conference on Content-based image and video retrieval, 2008: 465--474.
[5]
Cuixia Ma, Yongjin Liu, Hongan Wang, Dongxing Teng, Guozhong Dai. Sketch-based annotation and visualization in video authoring. IEEE Transactions on Multimedia, 2012, 14(4):1153--1165.
[6]
Yongjin Liu, Cuixia Ma, Qiufang Fu, Xiaolan Fu, Shengfeng Qin, Lexing Xie. A sketch-based approach for interactive organization of video clips. ACM Transactions on Multimedia Computing, Communications and Applications, 2014, 11(1).
[7]
Hu, R., James, S., Wang, T., Collomosse, J. Markov random fields for sketch based video retrieval. Proceedings of the 3rd ACM conference on International conference on multimedia retrieval, 2013: 279--286.
[8]
Yuan, J., Zha, Z. J., Zheng, Y. T., Wang, M., Zhou, X., Chua, T. S. Learning concept bundles for video search with complex queries. Proceedings of the 19th ACM international conference on Multimedia, 2011: 453--462.
[9]
Cross, A., Bayyapunedi, M., Cutrell, E., Agarwal, A., Thies, W. TypeRighting: combining the benefits of handwriting and typeface in online educational videos. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 2013: 793--796.
[10]
Tarashima, S., Irie, G., Tsutsuguchi, K., Arai, H., Taniguchi, Y. Fast image/video collection summarization with local clustering. Proceedings of the 21st ACM international conference on Multimedia, 2013:725--728.
[11]
Monserrat, T. J. K. P., Zhao, S., McGee, K., Pandey, A. V. NoteVideo: facilitating navigation of blackboard-style lecture videos. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2013:1139--1148.
[12]
Jain A K, Vailaya A. Shape-based retrieval: A case study with trademark image databases {J}. Pattern recognition, 1998, 31 (9): 1369--1390.
[13]
Di Sciascio E, Mongiello M. Query by sketch and relevance feedback for content-based image retrieval over the web {J}. Journal of Visual Languages & Computing, 1999, 10(6): 565--584.
[14]
Tseng K Y, Lin Y L, Chen Y H, et al. Sketch-based image retrieval on mobile devices using compact hash bits{C}, Proceedings of the 20th ACM international conference on Multimedia, 2012:913--916.
[15]
Chang S F, Chen W, Meng H J, et al. VideoQ: an automated content based video search system using visual cues{C}, Proceedings of the fifth ACM international conference on Multimedia. 1997: 313--324.
[16]
Hu R, Collomosse J P. Motion-sketch Based Video Retrieval Using a Trellis Levenshtein Distance{C}, ICPR. 2010: 121--124.
[17]
Craggs, B., Kilgallon Scott, M., Alexander, J. ThumbReels: query sensitive web video previews based on temporal, crowdsourced, semantic tagging. Proceedings of the 32nd annual ACM conference on Human factors in computing systems, 2014: 1217--1220.
[18]
Collomosse, J. P., McNeill, G., Qian, Y. Storyboard sketches for content based video retrieval. IEEE 12th International Conference on Computer Vision, 2009: 245--252.
[19]
Morikawa, C., de Silva, G. C. User interaction techniques for multimedia retrieval. Proceedings of the 2012 Joint International Conference on Human-Centered Computer Environments, 2012: 68--75.
[20]
Monroy-Hernandez, A., Hill, B. M., Gonzanlez-Rivero, J. Computers can't give credit: How automatic attribution falls short in an online remixing community. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2011: 3421--343.
- Exploring the Benefits of Text and Sketch in Video Retrieval of Complex Queries
Recommendations
Style-sensitive 3D model retrieval through sketch-based queries
Multimedia in technology enhanced learningTraditional sketch-based 3D model retrieval methods are content-based, which return the search results by ranking the geometric similarities among a free-hand drawing and 3D model candidates. These conventional methods do not consider personal drawing ...
Mixing realities for sketch retrieval in Virtual Reality
VRCAI '19: Proceedings of the 17th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in IndustryUsers within a Virtual Environment often need support designing the environment around them with the need to find relevant content while remaining immersed. We focus on the familiar sketch-based interaction to support the process of content placing and ...
Free-hand sketch based image and video retrieval
MM '10: Proceedings of the 18th ACM international conference on MultimediaWe present an overview of our work to date on a sketch based retrieval of image and video. We present a fast technique for extracting motion trajectories from videos and a Viterbi matching approach for retrieving video clips using free-hand sketched ...
Comments
Information & Contributors
Information
Published In
August 2015
185 pages
ISBN:9781450334822
DOI:10.1145/2801040
- General Chair:
- Takayuki Itoh,
- Program Chairs:
- Paolo Bottoni,
- Shigeo Takahashi
Copyright © 2015 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 August 2015
Check for updates
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
VINCI '15
VINCI '15: The 8th International Symposium on Visual Information Communication and Interaction
August 24 - 26, 2015
AA, Tokyo, Japan
Acceptance Rates
VINCI '15 Paper Acceptance Rate 12 of 32 submissions, 38%;
Overall Acceptance Rate 71 of 193 submissions, 37%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 76Total Downloads
- Downloads (Last 12 months)2
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in