Abstract
In this paper, we propose a new approach for reconstructing 3D structure from motion parallax. Instead of obtaining 3D structure from multi-view geometry or factorization, a Deep Neural Network (DNN) based method is proposed without assuming the camera model explicitly. In the proposed method, the targets are first split into connected 3D corners, and then the DNN regressor is trained to estimate the relative 3D structure of each corner from the target rotation. Finally, a temporal integration is performed to further improve the reconstruction accuracy. The effectiveness of the method is proved by a typical experiment of the Kinetic Depth Effect (KDE) in human visual system, in which the DNN regressor reconstructs the structure of a rotating 3D bent wire. The proposed method is also applied to reconstruct another two real targets. Experimental results on both synthetic and real images show that the proposed method is accurate and effective.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wallach, H., O’Connell, D.N.: The kinetic depth effect. J. Exp. Psychol. 45, 205–217 (1953)
Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vision 9, 137–154 (1992)
Shi, J., Tomasi, C.: Good features to track. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, pp. 593–600 (1994)
Palmer, S.E.: Vision Science: Photons to Phenomenology. MIT Press, Cambridge (1999)
Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Hilton Head Island, pp. 690–696 (2000)
Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment – a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) ICCV-WS 1999. LNCS, vol. 1883, p. 298. Springer, Heidelberg (2000)
Gruber, A., Weiss, Y.: Multibody factorization with uncertainty and missing data using the EM algorithm. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, pp. 707–714 (2004)
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18, 1527–1554 (2006)
Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3D. ACM Trans. Graph. 5, 835–846 (2006)
Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, pp. 225–234 (2007)
Saxena, A., Sun, M., Ng, A.Y.: Learning 3-D scene structure from a single still image. In: International Conference on Computer Vision Workshop, Rio de Janeiro, pp. 1–8 (2007)
Saxena, A., Schulte, J., Ng, A.Y.: Depth estimation using monocular and stereo cues. In: International Joint Conference on Artificial Intelligence, Hyderabad, pp, 2197–2203 (2007)
Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2, 1–127 (2009)
Ross, D.A., Tarlow, D., Zemel, R.S.: Learning articulated structure and motion. Int. J. Comput. Vision 88, 214–237 (2010)
Hedau, V., Hoiem, D., Forsyth, D.: Recovering free space of indoor scenes from a single image. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Rhode Island, pp. 2807–2814 (2012)
Xiao, J., Russell, B.C., Torralba, A.: Localizing 3D cuboids in single-view images. In: Advances in Neural Information Processing Systems, Lake Tahoe, pp. 746–754 (2012)
Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. Pattern Anal. Mach. Intell. 35, 1915–1929 (2013)
Fouhey, D.F., Gupta, A., Hebert, M.: Data-driven 3D primitives for single image understanding. In: International Conference on Computer Vision, Sydney, pp. 3392–3399 (2013)
Tanskanen, P., Kolev, K., Meier, L., Camposeco, F., Saurer, O., Pollefeys, M.: Live metric 3D reconstruction on mobile phones. In: International Conference on Computer Vision, Sydney, pp. 65–72 (2013)
Li, B., Shen, C., Dai, Y., Hengel, A., He, M.: Depth and surface normal estimation from monocular images using regression on deep features and hierarchical CRFs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, pp. 1119–1127 (2015)
Resch, B., Lensch, H.P.A., Wang, O., Pollefeys, M., Sorkine-Hornung, A.: Scalable Structure from Motion for Densely Sampled Videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, pp. 3936–3944 (2015)
Acknowledgement
The work was supported in part by the National Basic Research Program of China (2013CB329304), the “Twelfth Five-Year” National Science&Technology Support Program of China (No.2012BAI12B01), the Major Project of National Social Science Foundation of China (No.12&ZD119), the research special fund for public welfare industry of health (201202001) and National Natural Science Foundation of China (No.81170906).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Liu, W., Dou, H., Wu, X. (2015). Learning to Reconstruct 3D Structure from Object Motion. In: Arik, S., Huang, T., Lai, W., Liu, Q. (eds) Neural Information Processing. ICONIP 2015. Lecture Notes in Computer Science(), vol 9489. Springer, Cham. https://doi.org/10.1007/978-3-319-26532-2_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-26532-2_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26531-5
Online ISBN: 978-3-319-26532-2
eBook Packages: Computer ScienceComputer Science (R0)