Abstract
There is a strong demand for the digitization of books. To meet this demand, camera-based scanning systems are considered to be effective because they could work with the cameras built into mobile terminals. One promising technique proposed to speed up book digitization involves scanning a book while the user flips the pages. In this type of camera-based document image analysis, it is extremely important to rectify distorted images. In this paper, we propose a new method of reconstructing the 3D deformation and restoring a flat document image by utilizing a unique planar development property of a sheet of paper from a monocular image sequence captured while the paper is deformed. Our approach uses multiple input images and is based on the natural condition that a sheet of paper is a developable surface, enabling high-quality restoration without relying on the document structure. In the experiments, we tested the proposed method for the target application using images of different documents and different deformations, and demonstrated its effectiveness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Nakashima, T., Watanabe, Y., Komuro, T., Ishikawa, M.: Book flipping scanning. In: Adjunct Proceedings of UIST, pp. 79–80 (2009)
Carmo, M.P.D.: Differential Geometry of Curves and Surfaces. Prentice Hall (1976)
Watanabe, Y., Nakashima, T., Komuro, T., Ishikawa, M.: Estimation of non-rigid surface deformation using developable surface model. In: Proceedings of ICPR, pp. 197–200 (2010)
Cao, H., Ding, X., Liu, C.: Rectifying the bound document image captured by the camera: A model based approach. In: Proceedings of ICDAR, pp. 71–75 (2003)
Liang, J., DeMenthon, D., Doermann, D.: Unwarping Images of Curved Documents Using Global Shape Optimization. In: Proceedings of CBDAR, pp. 25–29 (2005)
Yamashita, A., Kawarago, A., Kaneko, T., Miura, K.T.: Shape reconstruction and image restoration for non-flat surfaces of documents with a stereo vision system. In: Proceedings of ICPR, pp. 482–485 (2004)
Zhang, Z., Tan, C.L., Fan, L.: Estimation of 3D shape of warped document surface for image restoration. In: Proceedings of ICPR, pp. 486–489 (2004)
Prados, E., Camilli, F.: A unifying and rigorous shape from shading method adapted to realistic data and applications. Journal of Mathematical Imaging and Vision 25, 307–328 (2006)
Hase, H., Shinokawa, T., Yoneda, M., Suen, C.: Recognition of rotated characters by eigen-space. In: Proceedings of ICDAR, pp. 731–735 (2003)
Narita, R., Ohyama, W., Wakabayashi, T., Kimura, F.: Three dimensional rotation-free recognition of characters. In: Proceedings of ICDAR, pp. 824–828 (2011)
Liang, J., DeMenthon, D., Doermann, D.: Geometric rectification of camera-captured document images. IEEE Transactions on Pattern Analysis and Machine Intelligence 30, 591–605 (2008)
Tian, Y., Narasimhan, S.G.: Rectification and 3D Reconstruction of Curved Document Images. In: Proceedings of ICCV, pp. 377–384 (2011)
Zhang, L., Tan, C.L.: Warped Image Restoration with Applications to Digital Libraries. In: Proceedings of ICDAR, pp. 192–196 (2005)
Gumerov, N.A., Zandifar, A., Duraiswami, R., Davis, L.S.: 3d structure recovery and unwarping of surfaces applicable to planes. International Journal of Computer Vision 66, 261–281 (2006)
Crum, W.R., Hartkens, T., Hill, D.L.G.: Non-rigid image registration: theory and practice. The British Journal of Radiology 77, 140–153 (2004)
Bartoli, A., Zisserman, A.: Direct estimation of non-rigid registrations. In: Proceedings of BMVC (2004)
Gay-Bellile, V., Bartoli, A., Sayd, P.: Direct estimation of nonrigid registrations with image-based self-occlusion reasoning. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 87–104 (2010)
Salzmann, M., Urtasun, R., Fua, P.: Local deformation models for monocular 3D shape recovery. In: Proceedings of CVPR (2008)
Salzmann, M., Pilet, J., Ilic, S., Fua, P.: Surface deformation models for non-rigid 3D shape recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 1481–1487 (2007)
Brunet, F., Hartley, R., Bartoli, A., Navab, N., Malgouyres, R.: Monocular Template-Based Reconstruction of Smooth and Inextensible Surfaces. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 52–66. Springer, Heidelberg (2011)
Varol, A., Salzmann, M., Tola, E., Fua, P.: Template-free monocular reconstruction of deformable surfaces. In: Proceedings of the ICCV, pp. 1811–1818 (2009)
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded Up Robust Features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)
Sprengel, R., Rohr, K., Stiehl, H.S.: Thin-plate spline approximation for image registration. In: Proceedings of the IEEE Engineering in Medicine and Biology Society, pp. 1190–1191 (1996)
Hore, A., Ziou, D.: Image quality metrics: PSNR vs. SSIM. In: Proceedings of ICPR, pp. 2366–2369 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shibayama, H., Watanabe, Y., Ishikawa, M. (2013). Reconstruction of 3D Surface and Restoration of Flat Document Image from Monocular Image Sequence. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds) Computer Vision – ACCV 2012. ACCV 2012. Lecture Notes in Computer Science, vol 7727. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37447-0_27
Download citation
DOI: https://doi.org/10.1007/978-3-642-37447-0_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-37446-3
Online ISBN: 978-3-642-37447-0
eBook Packages: Computer ScienceComputer Science (R0)