Abstract
In digital community applications, geo-coded multimedia data including spatial videos, speech, and geo-narratives are collected and utilized by community users and researchers from multiple fields. It is often preferred that these data can be captured, visualized, and explored directly on mobile phones and tablets interactively. In this paper, we present a Geo-Video Mobile Application (GVM App) that collects geo-coded multimedia data for experts to process and analyze over an interactive visual exploration. This mobile App integrates user interactivity, AI-based semantic image segmentation, and audio transcription for effective data extraction and utilization. Then visualization functions are designed to quickly present geographical, semantic, and street view visual information for knowledge discovery. The users of this tool can include community workers, teachers, and tourists, and also span across multiple social disciplines in digital humanity studies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abowd, G.D., Atkeson, C.G., Hong, J., Long, S., Kooper, R., Pinkerton, M.: CyberGuide: a mobile context-aware tour guide. Wirel. Netw. 3(5), 421–433 (1997)
Agrawala, M., Stolte, C.: Rendering effective route maps: improving usability through generalization. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 241–249 (2001)
Amiruzzaman, M., Curtis, A., Zhao, Y., Jamonnak, S., Ye, X.: Classifying crime places by neighborhood visual appearance and police geonarratives: a machine learning approach. J. Comput. Soc. Sci. 4, 1–25 (2021)
Baus, J., Cheverst, K., Kray, C.: A survey of map-based mobile guides. In: Meng, L., Reichenbacher, T., Zipf, A. (eds.) Map-Based Mobile Services, pp. 193–209. Springer, Heidelberg (2005). https://doi.org/10.1007/3-540-26982-7_13
Blumenstein, K., Niederer, C., Wagner, M., Schmiedl, G., Rind, A., Aigner, W.: Evaluating information visualization on mobile devices: gaps and challenges in the empirical evaluation design space. In: Proceedings of the Sixth Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization, pp. 125–132 (2016)
Chalmers, D., Sloman, M., Dulay, N.: Map adaptation for users of mobile systems. In: Proceedings of the 10th International Conference on World Wide Web, pp. 735–744 (2001)
Cheverst, K., Davies, N., Mitchell, K., Friday, A.: Experiences of developing and deploying a context-aware tourist guide: the guide project. In: Proceedings of the 6th Annual International Conference on Mobile Computing and Networking, pp. 20–31 (2000)
Curtis, A., Fagan, W.F.: Capturing damage assessment with a spatial video: an example of a building and street-scale analysis of tornado-related mortality in Joplin, Missouri, 2011. Ann. Assoc. Am. Geogr. 103(6), 1522–1538 (2013)
Curtis, A., Felix, C., Mitchell, S., Ajayakumar, J., Kerndt, P.R.: Contextualizing overdoses in Los Angeles’s skid row between 2014 and 2016 by leveraging the spatial knowledge of the marginalized as a resource. Ann. Am. Assoc. Geogr. 108(6), 1521–1536 (2018)
Dillemuth, J.: Map design evaluation for mobile display. Cartogr. Geogr. Inf. Sci. 32(4), 285–301 (2005)
Gartner, G., Uhlirz, S.: Cartographic concepts for realizing a location based UMTS service: Vienna city guide lol@. In: Proceedings of the Cartographic Conference, pp. 3229–3239 (2001)
Hafidh Firmansyah, M., Paul, A., Bhattacharya, D., Malik Urfa, G.: AI based embedded speech to text using deepspeech. arXiv e-prints, arXiv-2002 (2020)
Hampe, M., Anders, K.H., Sester, M.: MRDB applications for data revision and real-time generalisation. In: Proceedings of the 21st International Cartographic Conference, pp. 10–16 (2003)
Harrie, L., Sarjakoski, L.T., Lehto, L.: A variable-scale map for small-display cartography. Int. Arch. Photogrammetry Remote Sens. Spat. Inf. Sci. 34(4), 237–242 (2002)
Jamonnak, S., et al.: GeoVisuals: a visual analytics approach to leverage the potential of spatial videos and associated geonarratives. Int. J. Geogr. Inf. Sci. 34(11), 2115–2135 (2020)
Lei, X., Jiang, X., Wang, C.: Design and implementation of a real-time video stream analysis system based on FFMPEG. In: 2013 Fourth World Congress on Software Engineering, pp. 212–216. IEEE (2013)
Magnisalis, I.D., Demetriadis, S.N.: Mobile widgets to support peer interaction visualization. In: 2014 IEEE 14th International Conference on Advanced Learning Technologies, pp. 191–193. IEEE (2014)
Mills, J.W., Curtis, A., Kennedy, B., Kennedy, S.W., Edwards, J.D.: Geospatial video for field data collection. Appl. Geogr 30(4), 533–547 (2010). https://doi.org/10.1016/j.apgeog.2010.03.008. Climate Change and Applied Geography - Place, Policy, and Practice
Nissen, F., Hvas, A., Münster-Swendsen, J., Brodersen, L.: Small-display cartography. GiMoDig Scientific Report (2003)
Poslad, S., Laamanen, H., Malaka, R., Nick, A., Buckle, P., Zipl, A.: CRUMPET: creation of user-friendly mobile services personalised for tourism. In: IET Conference Proceedings, pp. 28–32(4) (2001)
Rauschenbach, U., Jeschke, S., Schumann, H.: General rectangular fisheye views for 2D graphics. Comput. Graph. 25(4), 609–617 (2001)
Reichenbacher, T.: SVG for adaptive visualisations in mobile situations. In: SVG Open Conference, Zürich (2002)
Roto, V., Popescu, A., Koivisto, A., Vartiainen, E.: Minimap: a web page visualization method for mobile phones. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 35–44 (2006)
Schmidt-Belz, B., Laamanen, H., Poslad, S., Zipf, A.: Location-based mobile tourist services: first user experiences. In: Proceedings of the International Conference on Information and Communication Technologies In Tourism, ENTER 2003: 10th International Conference on Information Technology in Tourism (2003)
Schuch, L., Curtis, A., Davidson, J.: Reducing lead exposure risk to vulnerable populations: a proactive geographic solution. Ann. Am. Assoc. Geogr. 107(3), 606–624 (2017)
Sester, M., Brenner, C.: Continuous generalization for visualization on small mobile devices. In: Sester, M., Brenner, C. (eds.) Developments in Spatial Data Handling, pp. 355–368. Springer, Heidelberg (2005). https://doi.org/10.1007/3-540-26772-7_27
Wang, W., Yang, J., You, X.: Combining ElasticFusion with PSPNet for RGB-D based indoor semantic mapping. In: 2018 Chinese Automation Congress (CAC), pp. 2996–3001. IEEE (2018)
Zelenka, J.: Information and communication technologies in tourism-influence, dynamics, trends. Ph.D. thesis, Technická univerzita v Liberci (2009)
Zipf, A., Malaka, R., et al.: Developing location based services for tourism: the service providers’ view. In: ENTER, vol. 2001, pp. 83–92. Citeseer (2001)
Acknowledgement
We thank the reviewers for helpful suggestions and comments. This work was partly supported by NSF Grant 1739491.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Bhati, D., Amiruzzaman, M., Jamonnak, S., Zhao, Y. (2022). Interactive Visualization and Capture of Geo-Coded Multimedia Data on Mobile Devices. In: Kim, JH., Singh, M., Khan, J., Tiwary, U.S., Sur, M., Singh, D. (eds) Intelligent Human Computer Interaction. IHCI 2021. Lecture Notes in Computer Science, vol 13184. Springer, Cham. https://doi.org/10.1007/978-3-030-98404-5_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-98404-5_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98403-8
Online ISBN: 978-3-030-98404-5
eBook Packages: Computer ScienceComputer Science (R0)