Skip to main content
Log in

Shape from shading for the digitization of curved documents

Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Document digitization is faster and more affordable using digital cameras than scanners. On the other hand, if we aim at extending the basic digital camera functionalities for such a purpose, post-processings will be of first importance, at least to improve the text legibility. In this paper, we address the specific problem of the virtual flattening of curved documents, as for example the pages of an opened book lying on its spine. In order to compute the document shape, we use the shape from shading technique and discuss why, in some cases, it is more suitable than other 3D single-view reconstruction techniques. We extend the seminal work by Wada et al. (Proceedings of the IAPR Workshop on machine vision and applications, Tokyo, Japan, pp. 591–594, 1992) and consecutive papers, reformulating the problem in terms of perspective shape from shading. Finally, we design a complete post-processing algorithm and test it on real images. Even if the documents are much curved, it is shown that the restored images are almost identical to scanned images of the flattened documents.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wada, T., Matsuyama, T.: Shape from shading on textured cylindrical surface restoring distorted scanner images of unfolded book surfaces. In: Proceedings of the IAPR Workshop on Machine Vision and Applications, pp. 591–594. Tokyo, Japan (1992)

  2. Kanungo, T., Haralick, R.M., Phillips, I.: Global and local document degradation models. In: Proceedings of the 2nd International Conference on Document Analysis and Recognition, pp. 730–734. Tsukuba, Japan (1993)

  3. Zhang R., Tsai P.S., Cryer J.E. and Shah M. (1999). Shape from Shading: a survey. IEEE Trans. Patt. Anal. Mach. Intell. 21(8): 690–706

    Article  Google Scholar 

  4. Durou, J.D., Falcone, M., Sagona, M.: A survey of numerical methods for shape from shading. Rapport de Recherche 2004-2-R, Institut de Recherche en Informatique de Toulouse, Toulouse, France (2004)

  5. Prados, E., Faugeras, O.: Perspective shape from shading and viscosity solutions. [44] 826–831

  6. Tankus, A., Sochen, N., Yeshurun, Y.: A new perspective [on] shape-from-shading. [44] 862–869

  7. Courteille, F., Crouzil, A., Durou, J.D., Gurdjos, P.: Towards shape from shading under realistic photographic conditions. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 2, pp. 277–280. Cambridge,UK (2004)

  8. Prados, E.: Application of the theory of the viscosity solutions to the shape from shading problem. Thèse de doctorat, Université de Nice—Sophia Antipolis, Nice, France (2004)

  9. Tankus, A., Sochen, N., Yeshurun, Y.: Reconstruction of images by perspective shape-from-shading. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 3, pp. 778–781 Cambridge, UK (2004)

  10. Tang Y.Y. and Tang C.Y. (1993). Image transformation approach to nonlinear shape restoration. IEEE Trans. Syst. Man Cybern. 23: 155–172

    Article  MATH  Google Scholar 

  11. Weng, Y., Zhu, Q.: Nonlinear shape restoration for document images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 568–573. San Francisco, California, USA (1996)

  12. Zhang, Z., Tan, C.L.: Recovery of distorted document images from bound volumes. In: Proceedings of the 6th International Conference on Document Analysis and Recognition, pp. 429–433. Seattle, Washington, USA (2001)

  13. Lavialle, O., Molines, X., Angella, F., Baylou, P.: Active contours network to straighten distorted text lines. In: Proceedings of the IEEE International Conference on Image Processing, vol. 3, pp. 748–751. Thessaloniki, Greece (2001)

  14. Wu, C.H., Agam, G.: Document image de-warping for text/graphics recognition. In: Proceedings of the Joint IAPR International Workshops on Syntactical and Structural Pattern Recognition and Statistical Pattern Recognition, pp. 348–357. Windsor, Canada (2002)

  15. Tsoi, Y.C., Brown, M.S.: Geometric and shading correction for images of printed materials: a unified approach using boundary. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 240–246 Washington, D.C., USA (2004)

  16. Brown M.S. and Seales W.B. (2004). Image restoration of arbitrarily warped documents. IEEE Trans. Patt. Anal. Mach. Intell. 26: 1295–1306

    Article  Google Scholar 

  17. Chambon, S., Crouzil, A.: Dense matching using correlation: new measures that are robust near occlusions. In: Proceedings of the British Machine Vision Conference, Norwich, Royaume-Uni, pp. 143–152 (2003)

  18. Yamashita, A., Kawarago, A., Kaneko, T., Miura, K.T.: Shape reconstruction and image restoration for non-flat surfaces of documents with a stereo vision system. In: Proceedings of the 17th International Conference on Pattern Recognition, vol. 1, pp. 482–485. Cambridge, UK (2004)

  19. Doncescu, A., Bouju, A., Quillet, V.: Former books digital processing: image warping. In: Proceedings of the IEEE Workshop on Document Image Analysis, pp. 5–9 San. Juan, Puerto Rico (1997)

  20. Brown, M.S., Seales, W.B.: Document restoration using 3D shape: a general deskewing algorithm for arbitrarily warped documents. In: Proceedings of the 8th IEEE International Conference on Computer Vision, vol. 1, pp. 367–375, Vancouver, Canada (2001)

  21. Brown, M.S., Seales, W.B.: Beyond 2D images: effective 3D imaging for library materials. In: Proceedings of the 5th ACM conference on digital libraries, pp. 27–36 San Antonio, TX, USA (2000)

  22. Sun, M., Yang, R., Yun, L., Landon, G., Seales, W.B., Brown, M.S.: Geometric and photometric restoration of distorted documents. In: Proceedings of the 10th IEEE International Conference on Computer Vision, vol. 2, pp. 1117–1123. China (2005)

  23. Cao, H., Ding, X., Liu, C.: Rectifying the bound document image captured by the camera: a model based approach. In: Proceedings of the 7th International Conference on Document Analysis and Recognition, vol. 1, pp. 71–75. UK (2003)

  24. Liang, J., DeMenthon, D., Doermann, D.: Flattening curved documents in images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 338–345. San Diego, California, USA (2005)

  25. Kashimura, M., Nakajima, T., Onda, N., Saito, H., Ozawa, S.: Practical introduction of image processing technology to digital archiving of rare books. In: Proceedings of the International Conference on Signal Processing Applications and Technology, Toronto, Canada, pp. 1025–1029 (1998)

  26. Courteille, F., Durou, J.D., Gurdjos, P.: Transform your digital camera into a flatbed scanner. In: Proceedings of the 9th European Conference on Computer Vision, 2nd Workshop on Applications of Computer Vision, Graz, Austria, pp. 40–48 (2006)

  27. Gumerov, N., Zandifar, A., Duraiswami, R., Davis, L.S.: Structure of applicable surfaces from single views. In: Proceedings of the 8th European Conference on Computer Vision, vol. 3. Lecture notes in Computer Science, vol. 3022, Prague, Czech Republic, pp. 482–496 (2004)

  28. Cho, S.I., Saito, H., Ozawa, S.: A divide-and-conquer strategy in shape from shading problem. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 413–419. San Juan, Puerto Rico (1997)

  29. Wada T., Ukida H. and Matsuyama T. (1997). Shape from shading with interreflections under a proximal light source: distortion-free copying of an unfolded book. Int. J. Comput. Vis. 24: 125–135

    Article  Google Scholar 

  30. Tan C.L., Zhang L., Zhang Z. and Xia T. (2006). Restoring warped document images through 3D shape modeling. IEEE Trans. Patt. Anal. Mach. Intell. 28: 195–208

    Article  Google Scholar 

  31. Pilu, M.: Undoing paper curl distortion using applicable surfaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 67–72. Kauai, Hawaii, USA (2001)

  32. Brown, M.S., Pisula, C.: Conformal deskewing of non-planar documents. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 998–1004. San Diego, California, USA (2005)

  33. Horn B.K.P (1977). Understanding image intensities. Artif. Intell. 8: 201–231

    Article  MATH  Google Scholar 

  34. Penna M.A. (1989). A shape from shading analysis for a single perspective image of a polyhedron. IEEE Trans. Patt. Anal. Mach. Intell. 11: 545–554

    Article  Google Scholar 

  35. Lee K.M. and Kuo C.C.J. (1994). Shape from shading with perspective projection. Comput. Vis. Graph. Image Process. Image Underst. 59: 202–212

    Article  Google Scholar 

  36. Hasegawa J.K. and Tozzi C.L. (1996). Shape from shading with perspective projection and camera calibration. Comput. Graph. 20: 351–364

    Article  Google Scholar 

  37. Samaras, D., Metaxas, D.N.: Coupled lighting direction and shape estimation from single images. In: Proceedings of the 7th IEEE International Conference on Computer Vision, vol. 2, pp. 868–874. Kerkyra, Greece (1999)

  38. Lions P.L., Rouy E. and Tourin A. (1993). Shape-from-shading, viscosity solutions and edges. Numer. Math. 64: 323–353

    Article  MATH  MathSciNet  Google Scholar 

  39. Horn, B.K.P., Brooks, M.J. (eds.): Shape from shading. MIT, Cambridge (1989)

  40. Horn B.K.P. and Brooks M.J. (1986). The variational approach to shape from shading. Computer Vision, Graph. Image Process. 33: 174–208

    Article  Google Scholar 

  41. Nayar S.K., Ikeuchi K. and Kanade T. (1991). Shape from interreflections. Int. J. Comput. Vis. 6: 173–195

    Article  Google Scholar 

  42. Fisher, R., Perkins, S., Walker, A.E.W.: Pixel Division. http://www.homepages.inf.ed.ac.uk/rbf/HIPR2/pixdiv.htm

  43. Smythe, D.B.: A two-pass mesh warping algorithm for object transformation and image interpolation. Technical Memo 1030, Industrial Light and Magic, Computer Graphics Department, Lucasfilm Ltd (1990)

  44. Proceedings of the 9th IEEE International Conference on Computer Vision, vol. 2. In: Proceedings of the 9th IEEE International Conference on Computer Vision, vol. 2, Nice, France (2003)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Frédéric Courteille.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Courteille, F., Crouzil, A., Durou, JD. et al. Shape from shading for the digitization of curved documents. Machine Vision and Applications 18, 301–316 (2007). https://doi.org/10.1007/s00138-006-0062-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-006-0062-y

Keywords

Navigation