Skip to main content
SpringerLink
Log in

GPU-accelerated uncapacitated facility location and semi-dense SymStereo pipelines for piecewise-planar-based 3D reconstruction

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Planar 3D reconstruction presents advantages over point cloud representations. This work focuses on the acceleration of piecewise-planar-based 3D reconstruction, a StereoScan method. We identify the SymStereo (logN) and uncapacitated facility location (UFL) algorithms as the most computationally expensive tasks, consuming nearly 80 × of total runtime, when detecting planes in a single stereo pair on a sequential CPU pipeline. Consequently, these algorithms have been parallelized using single- and multi-GPU architectures to perform significantly faster than previous sequential approaches. Experimental results show that accelerated parallel implementations of SymStereo (logN) can process up to 56 frames per second, achieving a speedup of 38 × against the sequential C implementation (Intel Core i7-4790k). The parallel version of the message-passing algorithm (max-sum) for the UFL problem processes up to five matrices per second and outperforms the sequential C baseline for computing UFL by 38 ×.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Alexiadis, D., Zarpalas, D., Daras, P.: Real-time, full 3-D reconstruction of moving foreground objects from multiple consumer depth cameras. IEEE Trans. Multimedia 15(2), 339–358 (2013). https://doi.org/10.1109/TMM.2012.2229264

    Article  Google Scholar 

  2. Aliakbarpour, H., Almeida, L., Menezes, P., Dias, J.: Multi-sensor 3D volumetric reconstruction using CUDA. 3D Research 2(4), 6 (2011). https://doi.org/10.1007/3DRes.04(2011)6

    Article  Google Scholar 

  3. Antunes, M., Barreto, J.P.: Semi-dense piecewise planar stereo reconstruction using SymStereo and PEARL. In: Second International Conference on3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), pp. 230–237. IEEE (2012)

  4. Antunes, M., Barreto, J.P.: SymStereo: stereo matching using induced symmetry. Int. J. Comput. Vis. 10, 1–22 (2014)

    MATH  Google Scholar 

  5. Antunes, M., Barreto, J.P., Nunes, U.: Piecewise-planar reconstruction using two views. Image Vis. Comput. 46, 47–63 (2016)

    Article  Google Scholar 

  6. Antunes, M.G.: Stereo Reconstruction using Induced Symmetry and 3D scene priors. Ph.D thesis, http://www2.isr.uc.pt/michel/files/final.pdf (2014)

  7. Bódis-Szomorú, A., Riemenschneider, H., Van Gool, L.: Fast, approximate piecewise-planar modeling based on sparse structure-from-motion and superpixels. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 469–476 (2014)

  8. Delong, A., Osokin, A., Isack, H.N., Boykov, Y.: Fast approximate energy minimization with label costs. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 2173–2180. IEEE (2010)

  9. Frigo, M., Johnson, S.G.: The design and implementation of FFTW3. Proc. IEEE 93(2), 216–231 (2005)

    Article  Google Scholar 

  10. Frome, A., Cheung, G., Abdulkader, A., Zennaro, M., Wu, B., Bissacco, A., Adam, H., Neven, H., Vincent, L.: Large-scale privacy protection in Google Street View. In: IEEE International Conference on Computer Vision, CVPR, pp. 2373–2380 (2009). https://doi.org/10.1109/ICCV.2009.5459413

  11. Gallup, D., Frahm, J.M., Pollefeys, M.: Piecewise planar and non-planar stereo for urban scene reconstruction. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 1418–1425. IEEE (2010)

  12. Gijsen, F.J., Schuurbiers, J.C., van de Giessen, A.G., Schaap, M., van der Steen, A.F., Wentzel, J.J.: 3D reconstruction techniques of human coronary bifurcations for shear stress computations. J. Biomech. 47(1), 39–43 (2014). https://doi.org/10.1016/j.jbiomech.2013.10.021

    Article  Google Scholar 

  13. Graca, C., Falcao, G., Figueiredo, I., Kumar, S.: Hybrid multi-GPU computing: accelerated kernels for segmentation and object detection with medical image processing applications. J. Real-Time Image Process. (2015). https://doi.org/10.1007/s11554-015-0517-3

    Article  Google Scholar 

  14. Graca, C., Falcao, G., Kumar, S., Figueiredo, I.: Cooperative use of parallel processing with time or frequency-domain filtering for shape recognition. In: Proceedings of the 22nd European Signal Processing Conference, EUSIPCO, pp. 2085–2089 (2014)

  15. Graca, C., Raposo, C., Barreto, J.P., Nunes, U., Falcao, G.: UrbanScan Website: 3D modeling of urban scenes. http://montecristo.co.it.pt/PPR_Rec/ (2016)

  16. Guenoun, B., Hajj, F.E., Biau, D., Anract, P., Courpied, J.P.: Reliability of a new method for evaluating femoral stem positioning after total hip arthroplasty based on stereoradiographic 3D reconstruction. J. Arthroplasty 30(1), 141–144 (2015). https://doi.org/10.1016/j.arth.2014.07.033

    Article  Google Scholar 

  17. Harris, M., et al.: Optimizing parallel reduction in CUDA. Nvidia Dev. Technol. 2(4), 70 (2007)

    Google Scholar 

  18. Hirschmuller, H., Scharstein, D.: Evaluation of cost functions for stereo matching. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 1–8. IEEE (2007)

  19. Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., Fitzgibbon, A.: KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology, UIST ’11, pp. 559–568. ACM, New York, NY, USA (2011). https://doi.org/10.1145/2047196.2047270

  20. Kalarot, R., Morris, J.: Implementation of symmetric dynamic programming stereo matching algorithm using CUDA pp. 141–146 (2010)

  21. Kou, W., Cheong, L.F., Zhou, Z.: Proximal robust factorization for piecewise planar reconstruction. Comput. Vis. Image Underst. 166, 88–101 (2018)

    Article  Google Scholar 

  22. Ladikos, A., Benhimane, S., Navab, N.: Efficient visual hull computation for real-time 3D reconstruction using CUDA. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW, pp. 1–8 (2008). https://doi.org/10.1109/CVPRW.2008.4563098

  23. Lazic, N., Frey, B.J., Aarabi, P.: Solving the uncapacitated facility location problem using message passing algorithms. In: International Conference on Artificial Intelligence and Statistics, pp. 429–436 (2010)

  24. Li, J., Sun, J., Song, Y., Zhao, J.: Accelerating MRI reconstruction via three-dimensional dual-dictionary learning using CUDA. J. Supercomput. 71(7), 2381–2396 (2015). https://doi.org/10.1007/s11227-015-1386-z

    Article  Google Scholar 

  25. Luo, Y., Duraiswami, R.: Canny edge detection on Nvidia CUDA. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW, pp. 1–8 (2008). https://doi.org/10.1109/CVPRW.2008.4563088

  26. Mei, X., Sun, X., Zhou, M., Jiao, S., Wang, H., Zhang, X.: On building an accurate stereo matching system on graphics hardware. In: IEEE International Conference on Computer Vision Workshops, ICCV Workshops, pp. 467–474 (2011). https://doi.org/10.1109/ICCVW.2011.6130280

  27. Melo, R., Barreto, J.P., Falcao, G.: A new solution for camera calibration and real-time image distortion correction in medical endoscopy - initial technical evaluation. IEEE Trans. Biomed. Eng. 59(3), 634–644 (2012). https://doi.org/10.1109/TBME.2011.2177268

    Article  Google Scholar 

  28. Micusik, B., Kosecka, J.: Piecewise planar city 3d modeling from street view panoramic sequences. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR, pp. 2906–2912 (2009). https://doi.org/10.1109/CVPR.2009.5206535

  29. Nvidia, C.: cuBLAS. [Online]. https://developer.nvidia.com/cuBLAS (2015)

  30. Nvidia, C.: cuFFT. [Online]. https://developer.nvidia.com/cuFFT (2015)

  31. Nvidia, C.: CUDA C best practices guide. Available: https://docs.nvidia.com/cuda/cuda-c-best-practices-guide/index.html#device-memory-spaces (2019)

  32. Ogawa, K., Ito, Y., Nakano, K.: Efficient canny edge detection using a GPU, pp. 279–280 (2010). https://doi.org/10.1109/IC-NC.2010.13

  33. Park, I.K., Singhal, N., Lee, M.H., Cho, S., Kim, C.: Design and Performance Evaluation of Image Processing Algorithms on GPUs. IEEE Trans. Parallel Distrib. Syst. 22(1), 91–104 (2011). https://doi.org/10.1109/TPDS.2010.115

    Article  Google Scholar 

  34. Podlozhnyuk, V., Harris, M., Young, E.: Nvidia CUDA C programming guide. Nvidia Corporation (2012)

  35. Pollefeys, M., Nistér, D., Frahm, J.M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim, S.J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewénius, H., Yang, R., Welch, G., Towles, H.: Detailed real-time urban 3D reconstruction from video. Int. J. Comput. Vis. 78(2–3), 143–167 (2008). https://doi.org/10.1007/s11263-007-0086-4

    Article  Google Scholar 

  36. Ralha, R., Falcao, G., Amaro, J., Mota, V., Antunes, M., Barreto, J., Nunes, U.: Parallel refinement of slanted 3D reconstruction using dense stereo induced from symmetry. Journal of Real-Time Image Processing pp. 1–19 (2016). https://doi.org/10.1007/s11554-016-0592-0

  37. Raposo, C., Antunes, M., Barreto, J.: Piecewise-planar stereoscan:structure and motion from plane primitives. In: D. Fleet, T. Pajdla, B. Schiele, T. Tuytelaars (eds.) European Conference on Computer Vision, ECCV, Lecture Notes in Computer Science, vol. 8690, pp. 48–63. Springer International Publishing (2014). https://doi.org/10.1007/978-3-319-10605-2_4

  38. Raposo, C., Antunes, M., Barreto, J.P.: Piecewise-Planar StereoScan: sequential structure and motion using plane primitives. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2017)

  39. Raposo, C., Barreto, J.P.: \(\pi\)Match: Monocular vSLAM and Piecewise Planar Reconstruction using Fast Plane Correspondences. In: European Conference on Computer Vision, ECCV, pp. 380–395. Springer (2016)

  40. Salmen, J., Houben, S., Schlipsing, M.: Google street view images support the development of vision-based driver assistance systems. In: IEEE Intelligent Vehicles Symposium, IV, pp. 891–895. IEEE (2012)

  41. Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, vol. 1, pp. 519–528. IEEE (2006)

  42. Sinha, S.N., Steedly, D., Szeliski, R.: Piecewise planar stereo for image-based rendering. In: IEEE International Conference on Computer Vision Workshops, ICCV, pp. 1881–1888 (2009)

  43. Torii, A., Havlena, M., Pajdla, T.: From Google street view to 3d city models. In: IEEE International Conference on Computer Vision Workshops, ICCV Workshops, pp. 2188–2195 (2009). https://doi.org/10.1109/ICCVW.2009.5457551

  44. Vineet, V., Narayanan, P.: CUDA cuts: Fast graph cuts on the GPU. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPRW, pp. 1–8 (2008). https://doi.org/10.1109/CVPRW.2008.4563095

  45. Vogel, C., Roth, S., Schindler, K.: View-consistent 3d scene flow estimation over multiple frames. In: European Conference on Computer Vision, ECCV, pp. 263–278. Springer (2014)

  46. Whitmarsh, T., Humbert, L., Barquero, L.M.D.R., Gregorio, S.D., Frangi, A.F.: 3D reconstruction of the lumbar vertebrae from anteroposterior and lateral dual-energy X-ray absorptiometry. Medical Image Analysis 17(4), 475–487 (2013). https://doi.org/10.1016/j.media.2013.02.002. http://www.sciencedirect.com/science/article/pii/S1361841513000091

  47. Woodford, O., Torr, P., Reid, I., Fitzgibbon, A.: Global stereo reconstruction under second-order smoothness priors. IEEE Trans. Pattern Anal. Mach. Intell. 31(12), 2115–2128 (2009)

    Article  Google Scholar 

  48. Yang, Z., Zhu, Y., Pu, Y.: Parallel image processing based on CUDA. Int. Conf. Comput. Sci. Softw. Eng. 3, 198–201 (2008). https://doi.org/10.1109/CSSE.2008.1448

    Article  Google Scholar 

  49. Yeom, E., Nam, K.H., Jin, C., Paeng, D.G., Lee, S.J.: 3D reconstruction of a carotid bifurcation from 2D transversal ultrasound images. Ultrasonics 54(8), 2184–2192 (2014)

    Article  Google Scholar 

  50. Zamir, A.R., Shah, M.: Accurate image localization based on google maps street view. In: European Conference on Computer Vision, ECCV, pp. 255–268. Springer (2010)

  51. Zhang, K., Lu, J., Yang, Q., Lafruit, G., Lauwereins, R., Van Gool, L.: Real-time and accurate stereo: A scalable approach with bitwise fast voting on CUDA. IEEE Trans. Circuits Syst. Video Technol. 21(7), 867–878 (2011). https://doi.org/10.1109/TCSVT.2011.2133150

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially supported by a Google Faculty Research Award from Google Inc. and also by the Portuguese Foundation for Science and Technology (FCT) under grant AMS-HMI12: RECI/EEIAUT/0181/2012. It was equally supported by Instituto de Telecomunicações and funded by FCT/MCTES through national funds and when applicable co-funded EU funds under the project UIDB/EEA/50008/2020. It was carried out at the Multimedia Signal Processing Laboratory, a GPU Research Center from the University of Coimbra, Portugal.

Author information

Authors and Affiliations

  1. University of Coimbra, Instituto de Telecomunicações, Department of Electrical and Computer Engineering, 3030-290, Coimbra, Portugal

    Carlos Graca & Gabriel Falcao

  2. University of Coimbra, Institute of Systems and Robotics, Department of Electrical and Computer Engineering, 3030-290, Coimbra, Portugal

    Carolina Raposo, Joao P. Barreto & Urbano Nunes

Authors
  1. Carlos Graca
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carolina Raposo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Joao P. Barreto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Urbano Nunes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Gabriel Falcao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gabriel Falcao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Graca, C., Raposo, C., Barreto, J.P. et al. GPU-accelerated uncapacitated facility location and semi-dense SymStereo pipelines for piecewise-planar-based 3D reconstruction. J Real-Time Image Proc 18, 445–461 (2021). https://doi.org/10.1007/s11554-020-00974-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-020-00974-z

Keywords

Search

Navigation