Abstract
Haptic interaction requires the content creators to make haptic models of the virtual objects while it is not always possible or feasible, especially when it comes to using real images or videos as elements of interaction. We, therefore, propose tangible images and image-driven haptic rendering where a displayed image is used as a source of the force-feedback calculations at any pixel touched by the haptic device. We introduce the main idea and describe how it is implemented as a core algorithm for image-driven haptic rendering, as well as for a few particular cases of haptic rendering emphasizing colors, contours and textures of the objects displayed in the images. Implementations of the proposed method to desktop tangible image application and haptic video communication on the web are presented as a proof of concept.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Rasool, S., Sourin, A.: Image-Driven Haptic Rendering in Virtual Environments. In: 2013 International Conference on Cyberworlds, CW (2013)
Durou, J.-D., Falcone, M., Sagona, M.: Numerical methods for shape-from-shading: A new survey with benchmarks. Computer Vision and Image Understanding 109(1), 22–43 (2008)
Zhang, R., et al.: Shape from Shading: A Survey. IEEE Trans. Pattern Anal. Mach. Intell. 21(8), 690–706 (1999)
Breuss, M., et al.: Numerical algorithms for perspective shape from shading. Kybernetika 46(2), 207–225 (2010)
Ecker, A., Jepson, A.D.: Polynomial shape from shading. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, USA (2010)
Alexa, M., Matusik, W.: Reliefs as images. In: ACM SIGGRAPH 2010 Papers, pp. 1–7. ACM, Los Angeles (2010)
Longuet-Higgins, H.C.: A computer algorithm for reconstructing a scene from two projections. Nature 293(5828), 133–135 (1981)
Szeliski, R.: Stereo correspondence. In: Computer Vision, pp. 467–503. Springer, London (2011)
Vogiatzis, G., Hernández, C.: Practical 3D Reconstruction Based on Photometric Stereo. In: Cipolla, R., Battiato, S., Farinella, G. (eds.) Computer Vision. SCI, vol. 285, pp. 313–345. Springer, Heidelberg (2010)
Szeliski, R.: 3D reconstruction. In: Computer Vision, pp. 505–541. Springer, London (2011)
Li, J., Song, A., Zhang, X.: Haptic Texture Rendering Using Single Texture Image. In: 2010 International Symposium on Computational Intelligence and Design, ISCID (2010)
Shi, Y., Pai, D.K.: Haptic display of visual images. In: IEEE Virtual Reality Annual International Symposium, Albuquerque, USA (1997)
Basdogan, C.: From 2D images to 3D tangible models: autostereoscopic and haptic visualization of Martian rocks in virtual environments. Presence: Teleoper. Virtual Environ. 16(1), 1–15 (2007)
Di Stefano, L., Mattoccia, S.: Real-Time Stereo within the VIDET Project. Real-Time Imaging 8(5), 439–453 (2002)
Kim, S.-C., Kyung, K.-U., Kwon, D.S.: Haptic annotation for an interactive image. In: 5th ACM International Conference on Ubiquitous Information Management and Communication, ICUIMC 2011. ACM, Seoul (2011)
Kagawa, T., et al.: A Study of Haptic Interaction for Image Edition Tools. In: International Conference on Complex, Intelligent and Software Intensive Systems, Krakow, Poland (2010)
Lareau, D., Jochen, L.: Haptic rendering of photographs. In: 2012 IEEE International Workshop on Haptic Audio Visual Environments and Games, HAVE (2012)
Pai, D.K., Reissell, L.M.: Haptic interaction with multiresolution image curves. Computers & Graphics 21(4), 405–411 (1997)
Krufka, S.E., Barner, K.E., Aysal, T.C.: Visual to Tactile Conversion of Vector Graphics. IEEE Transactions on Neural Systems and Rehabilitation Engineering 15(2), 310–321 (2007)
Wang, Q.: Translation of graphics to haptic boundary representation, Department of Electrical and Computer Engineering, McGill University, p. 65 (1999)
Way, T.P., Barner, K.E.: Automatic visual to tactile translation. II. Evaluation of the TACTile image creation system. IEEE Transactions on Rehabilitation Engineering 5(1), 95–105 (1997)
Kim, S.-C., Kwon, D.-S.: Haptic interaction with objects in a picture based on pose estimation. In: Multimedia Tools and Applications, pp. 1–22 (2013)
Li, J., Song, A., Zhang, X.: Image-based haptic texture rendering. In: Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and its Applications in Industry, pp. 237–242. ACM, Seoul (2010)
Vasudevan, H., Manivannan, M.: Tangible Images: Runtime Generation of Haptic Textures From Images. In: Proceedings of the 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 357–360. EEE Computer Society, Reno (2008)
Adi, W., Sulaiman, S.: Haptic Texture Rendering Based on Visual Texture Information: A Study to Achieve Realistic Haptic Texture Rendering. In: Badioze Zaman, H., Robinson, P., Petrou, M., Olivier, P., Schröder, H., Shih, T.K. (eds.) IVIC 2009. LNCS, vol. 5857, pp. 279–287. Springer, Heidelberg (2009)
Juan, W., Aiguo, S., Chuiguo, Z.: A novel haptic texture display based on image processing. In: IEEE International Conference on Robotics and Biomimetics, ROBIO 2007, Sanya, China (2007)
Xu, S., et al.: An improved switching vector median filter for image-based haptic texture generation. In: 2012 5th International Congress on Image and Signal Processing, CISP (2012)
O’Modhrain, S., Oakley, I.: Adding interactivity: active touch in broadcast media. In: Proceedings of the 12th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, HAPTICS 2004, Chicago, USA (2004)
Mee Young, S., et al.: Touchable Video and Tactile Audio. In: 11th IEEE International Symposium on Multimedia, ISM 2009, San Diego, USA (2009)
Cha, J., et al.: 3D video player system with haptic interaction based on depth image-based representation. IEEE Transactions on Consumer Electronics 52(2), 477–484 (2006)
Gaw, D., Morris, D., Salisbury, K.: Haptically Annotated Movies: Reaching Out and Touching the Silver Screen. In: 14th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Alexandria, USA (2006)
Bayakovski, Y., et al.: Depth image-based representations for static and animated 3D objects. In: Proceedings of the International Conference on Image Processing 2002, Rochester, USA (2002)
Cha, J., Eid, M., Saddik, A.E.: Touchable 3D video system. ACM Trans. Multimedia Comput. Commun. Appl., TOMMCAP 5(4), 1–25 (2009)
Rasool, S., Sourin, A.: Tangible images. In: SIGGRAPH Asia 2011 Sketches. ACM, Hong Kong (2011)
Rasool, S., Sourin, A.: Haptic interaction with 2D images. In: Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry, pp. 13–22. ACM, Hong Kong (2011)
Rasool, S., Sourin, A.: Image-driven virtual simulation of arthroscopy. The Visual Computer 29(5), 333–344 (2013)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Rasool, S., Sourin, A. (2014). Image-Driven Haptic Rendering. In: Gavrilova, M.L., Tan, C.J.K., Mao, X., Hong, L. (eds) Transactions on Computational Science XXIII. Lecture Notes in Computer Science, vol 8490. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-43790-2_4
Download citation
DOI: https://doi.org/10.1007/978-3-662-43790-2_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-43789-6
Online ISBN: 978-3-662-43790-2
eBook Packages: Computer ScienceComputer Science (R0)