Skip to main content
SpringerLink
Log in

Embedded multi-processor system-on-programmable chip for smart camera pose estimation using nonlinear optimization methods

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

The PanoraMOS prototype is a complete localization system targeting Simultaneous Localization and Mapping applications. It is a panoramic camera that uses a single rotating linear sensor to capture cylindrical panoramic images at up to 3 frames per second. A complete localization algorithm has been implemented into the hardware architecture of the system. It has the ability to estimate its 3D pose in an indoor or an outdoor environment. This estimation is performed using a feature extractor and the Levenberg–Marquardt (LM) algorithm with the Random Sample Consensus (RANSAC) algorithm to perform detection. In this paper, we present the whole system particularly emphasize the localization algorithm and its implementation on a hardware architecture which is our main contribution. The implementation was done on a Multi-Processor System-on-Chip architecture. We present both software and hardware implementations with performance results on an ALTERA System-on-Programmable Chip target. The experimental results including processing times and application speed up show that our homogeneous network of processors is efficient for embedding the proposed image processing application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

References

  1. More, J.: The Levenberg-Marquardt algorithm: Implementation and theory. In: Numerical analysis, vol. 630, pp. 105–116 (1978). doi:10.1007/bfb0067700

  2. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting, with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  3. Durrant-Whyte, H., Bailey, T.: Simultaneous localization and mapping: Part I. IEEE Robot. Autom. Mag. 13, 99–108 (2006)

    Article  Google Scholar 

  4. Chenini, H., Derutin, J. P., Aufrere, R., Chapuis, R.: Parallel Embedded Processor Architecture for FPGA Based Image Processing using Parallel Software Skeletons. EURASIP J. Adv. Signal Process (2013)

  5. Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F., Sayd, P.: Real time localization and 3d reconstruction. In: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR’06, vol. 1, pp. 363–370. IEEE Computer Society, Washington, DC, USA, (2006). doi:10.1109/CVPR.2006.236

  6. Izo, T., Eric, W., Grimson, L.: Simultaneous pose estimation and camera calibration from multiple views. In: 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, ISSN 1:14, pp. 1063–6919 (2004). doi:10.1109/CVPR.2004.439

  7. Fischler, Martin A., Bolles, Robert C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  8. Andriluka, M., Roth, S., Schiele, B.: Monocular 3d pose estimation and tracking by detection. In: IEEE, CVPR, pp. 623–630 (2010)

  9. Didier, J.-Y., Ababsa, F., Mallem, M.: Hybrid camera pose estimation combining square fiducials localization technique and orthogonal iteration algorithm. Int. J. Image Graph. 8(1), 169–188 (2008)

    Article  Google Scholar 

  10. Di Stefano, Luigi, Mattoccia, Stefano, Tombari, Federico: Zncc-based template matching using bounded partial correlation. Pattern Recogn. Lett. 26(14), 2129–2134 (2005). doi:10.1016/j.patrec.2005.03.022. ISSN 0167–8655

    Article  Google Scholar 

  11. Förstner, W.: A framework for low level feature extraction. Computer Vision ECCV’94, pp. 383–394. Springer (1994)

  12. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1615–1630 (2005)

    Article  Google Scholar 

  13. Moravec, H.P.: Obstacle avoidance and navigation in the real world by a seeing robot rover. Obotics Institute, Stanford University. September (1980)

  14. Harris, C., Stephens.: A combined corner and edge detector. In: Proceedings of the 4th Alvey Vision Conference, pp. 147–151

  15. Li, Y.: FPGA Implementation for Image Processing Algorithms. Digital Signal Processing, Citeseer (2006)

  16. Dietrich, B.: Design and implementation of an FPGA-based stereo vision system for the EyeBot M6. University of Western Australia (2009)

  17. Metz, B., Morfopoulos, A.: Rapid corner detection using fpgas. National Aeronautics and Space Administration, December (2010)

  18. Goshorn, D., Cho, J., Kastner, R., Mirzaei, S.: Field programmable gate array implementation of parts-based object detection for real time video applications. In: 2010 International Conference on Field Programmable Logic and Applications (FPL), pp. 582–587

  19. Birem, M., Berry, F.: Hardware Architecture for Visual Feature Extraction. IEEE/RSJ Int. Conf. Intell. Robot. Syst. (2012)

Download references

Author information

Authors and Affiliations

  1. Aubiere Cedex, France

    Frantz Pelissier, Hanen Chenini, François Berry, Alexis Landrault & Jean-Pierre Derutin

Authors
  1. Frantz Pelissier
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hanen Chenini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. François Berry
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alexis Landrault
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jean-Pierre Derutin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Frantz Pelissier.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pelissier, F., Chenini, H., Berry, F. et al. Embedded multi-processor system-on-programmable chip for smart camera pose estimation using nonlinear optimization methods. J Real-Time Image Proc 12, 663–679 (2016). https://doi.org/10.1007/s11554-014-0454-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-014-0454-6

Keywords

Search

Navigation