Skip to main content
SpringerLink
Log in

Dynamic Real-Time Spatio-Temporal Acquisition and Rendering in Adverse Environments

  • Conference paper
  • First Online:
Geographical Information Systems Theory, Applications and Management (GISTAM 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2107))

  • 14 Accesses

Abstract

This paper introduces NausicaaVR, a novel hardware/software system designed to acquire and render intricate 3D environments, with a particular emphasis on challenging and adverse contexts. In doing so, we navigate the complex landscape of system calibration and rendering, while seamlessly integrating data from multiple sensors. We explore the distinctive challenges inherent in adverse environments, juxtaposing them against conventional automotive scenarios. Through a comprehensive exposition of all constituent elements of the NausicaaVR system, we offer transparent insights into the encountered obstacles and the intricate decisions that were instrumental in surmounting them. This study seeks to illuminate the developmental trajectory of NausicaaVR and analogous systems, thereby furnishing a repository of knowledge and understanding poised to benefit future research and the pragmatic implementation of such cutting-edge technologies.

NAUSICAA- NAUtical Safety by means of Integrated ComputerAssisted Appliances 4.0 (DIT.AD004.136).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. An, P., et al.: Geometric calibration for lidar-camera system fusing 3D–2D and 3D–3D point correspondences. Opt. Express 28(2), 2122–2141 (2020). https://doi.org/10.1364/OE.381176. https://opg.optica.org/oe/abstract.cfm?URI=oe-28-2-2122

  2. Berger, M., et al.: A survey of surface reconstruction from point clouds. In: Computer Graphics Forum, vol. 36, pp. 301–329. Wiley Online Library (2017)

    Google Scholar 

  3. Brinkmann, M., Hahn, A.: Testbed architecture for maritime cyber physical systems. In: 2017 IEEE 15th International Conference on Industrial Informatics (INDIN), pp. 923–928 (2017). https://doi.org/10.1109/INDIN.2017.8104895

  4. Bu, Z., Sun, C., Wang, P., Dong, H.: Calibration of camera and flash lidar system with a triangular pyramid target. Appl. Sci. 11(2) (2021). https://doi.org/10.3390/app11020582. https://www.mdpi.com/2076-3417/11/2/582

  5. Callieri, M., Cignoni, P., Corsini, M., Scopigno, R.: Masked photo blending: mapping dense photographic dataset on high-resolution 3D models. Comput. Graph. 32(4), 464–473 (2008). http://vcg.isti.cnr.it/Publications/2008/CCCS08, for the online version: http://dx.doi.org/10.1016/j.cag.2008.05.004

  6. Chen, Q., Xie, Y., Guo, S., Bai, J., Shu, Q.: Sensing system of environmental perception technologies for driverless vehicle: a review of state of the art and challenges. Sens. Actuators A 319, 112566 (2021). https://doi.org/10.1016/j.sna.2021.112566. https://www.sciencedirect.com/science/article/pii/S0924424721000273

  7. Corsini, M., Cignoni, P., Scopigno, R.: Efficient and flexible sampling with blue noise properties of triangular meshes. IEEE Trans. Visual Comput. Graphics 18(6), 914–924 (2012). https://doi.org/10.1109/TVCG.2012.34

    Article  Google Scholar 

  8. dSPACE (2021). https://www.dspace.com/en/pub/home/applicationfields/stories/smartkai-parking-assistance-f.cfm

  9. Dutta, S., Ganovelli, F., Cignoni, P.: On-the-fly acquisition and rendering with low cost lidar and RGB cameras for marine navigation. In: Grueau, C., Rodrigues, A., Ragia, L. (eds.) Proceedings of the 9th International Conference on Geographical Information Systems Theory, Applications and Management, GISTAM 2023, Prague, Czech Republic, 25–27 April 2023, pp. 176–183. SCITEPRESS (2023). https://doi.org/10.5220/0011855000003473

  10. Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F., Marín-Jiménez, M.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 47(6), 2280–2292 (2014). https://doi.org/10.1016/j.patcog.2014.01.005. https://www.sciencedirect.com/science/article/pii/S0031320314000235

  11. Gong, X., Lin, Y., Liu, J.: 3D LIDAR-camera extrinsic calibration using an arbitrary trihedron. Sensors 13(2), 1902–1918 (2013). https://www.mdpi.com/1424-8220/13/2/1902

  12. Grammatikopoulos, L., Papanagnou, A., Venianakis, A., Kalisperakis, I., Stentoumis, C.: An effective camera-to-lidar spatiotemporal calibration based on a simple calibration target. Sensors 22(15) (2022). https://www.mdpi.com/1424-8220/22/15/5576

  13. Hahn, T., Damerius, R., Rethfeldt, C., Schubert, A.U., Kurowski, M., Jeinsch, T.: Automated maneuvering using model-based control as key to autonomous shipping. at - Automatisierungstechnik 70(5), 456–468 (2022). https://doi.org/10.1515/auto-2021-0146

  14. ImagingSource (2017). https://www.theimagingsource.com

  15. Johnson, S.G.: The NLopt nonlinear-optimization package (2007). https://github.com/stevengj/nlopt

  16. Kang, J., Doh, N.L.: Automatic targetless camera-LIDAR calibration by aligning edge with gaussian mixture model. J. Field Robot. 37(1), 158–179 (2020). https://doi.org/10.1002/rob.21893. https://onlinelibrary.wiley.com/doi/abs/10.1002/rob.21893

  17. Kim, H., Kim, D., Park, B., Lee, S.M.: Artificial intelligence vision-based monitoring system for ship berthing. IEEE Access 8, 227014–227023 (2020). https://doi.org/10.1109/ACCESS.2020.3045487

    Article  Google Scholar 

  18. Kodaira, A., Zhou, Y., Zang, P., Zhan, W., Tomizuka, M.: SST-Calib: simultaneous spatial-temporal parameter calibration between lidar and camera, pp. 2896–2902. IEEE Press (2022). https://doi.org/10.1109/ITSC55140.2022.9922085

  19. Li, Q., Queralta, J.P.n., Gia, T.N., Zou, Z., Westerlund, T.: Multi-sensor fusion for navigation and mapping in autonomous vehicles: accurate localization in urban environments. Unmanned Syst. 08(03), 229–237 (2020). https://doi.org/10.1142/S2301385020500168

  20. Li, X., He, F., Li, S., Zhou, Y., Xia, C., Wang, X.: Accurate and automatic extrinsic calibration for a monocular camera and heterogenous 3D lidars. IEEE Sens. J. 22(16), 16472–16480 (2022). https://doi.org/10.1109/JSEN.2022.3189041

    Article  Google Scholar 

  21. Martelli, M., Virdis, A., Gotta, A., Cassarà, P., Di Summa, M.: An outlook on the future marine traffic management system for autonomous ships. IEEE Access 9, 157316–157328 (2021). https://doi.org/10.1109/ACCESS.2021.3130741

    Article  Google Scholar 

  22. Moghadam, P., Bosse, M., Zlot, R.: Line-based extrinsic calibration of range and image sensors. In: 2013 IEEE International Conference on Robotics and Automation, pp. 3685–3691 (2013). https://doi.org/10.1109/ICRA.2013.6631095

  23. Nowicki, M.R.: Spatiotemporal calibration of camera and 3D laser scanner. IEEE Robot. Autom. Lett. 5, 6451–6458 (2020)

    Article  Google Scholar 

  24. NVidia (2001). https://www.nvidia.com/en-us/drivers/Projective-Texture-Mapping/

  25. NVIDIA: NVIDIA announces Jetson TX2: Parker comes to NVIDIA’s embedded system kit (2017)

    Google Scholar 

  26. Paneque, J., Valseca, V., Martínez-de Dios, J.R., Ollero, A.: Autonomous reactive lidar-based mapping for powerline inspection. In: 2022 International Conference on Unmanned Aircraft Systems (ICUAS), pp. 962–971 (2022). https://doi.org/10.1109/ICUAS54217.2022.9836213

  27. Park, C., Moghadam, P., Kim, S., Sridharan, S., Fookes, C.: Spatiotemporal camera-lidar calibration: a targetless and structureless approach. IEEE Robot. Autom. Lett. 5(2), 1556–1563 (2020). https://doi.org/10.1109/LRA.2020.2969164

    Article  Google Scholar 

  28. Park, Y., Yun, S., Won, C.S., Cho, K., Um, K., Sim, S.: Calibration between color camera and 3D lidar instruments with a polygonal planar board. Sensors 14(3), 5333–5353 (2014). https://doi.org/10.3390/s140305333. https://www.mdpi.com/1424-8220/14/3/5333

  29. Perera, L., Moreira, L., Santos, F., Ferrari, V., Sutulo, S., Soares, C.G.: A navigation and control platform for real-time manoeuvring of autonomous ship models. IFAC Proc. Vol. 45(27), 465–470 (2012). https://doi.org/10.3182/20120919-3-IT-2046.00079. https://www.sciencedirect.com/science/article/pii/S1474667016312733, 9th IFAC Conference on Manoeuvring and Control of Marine Craft

  30. Peršić, J., Petrović, L., Marković, I., Petrović, I.: Spatiotemporal multisensor calibration via gaussian processes moving target tracking. IEEE Trans. Rob. 37(5), 1401–1415 (2021). https://doi.org/10.1109/TRO.2021.3061364

    Article  Google Scholar 

  31. Pizer, S.M., et al.: Adaptive histogram equalization and its variations. Comput. Vis. Graph. Image Process. 39(3), 355–368 (1987)

    Article  Google Scholar 

  32. Powell, M.J.D.: The NEWUOA software for unconstrained optimization without derivatives. In: Pillo, G.D., Roma, M. (eds.) Large-Scale Nonlinear Optimization, Nonconvex Optimization and Its Applications, vol. 83, pp. 255–297. Springer, Boston (2006). https://doi.org/10.1007/0-387-30065-1_16

    Chapter  Google Scholar 

  33. Rehder, J., Beardsley, P., Siegwart, R., Furgale, P.: Spatio-temporal laser to visual/inertial calibration with applications to hand-held, large scale scanning. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 459–465 (2014). https://doi.org/10.1109/IROS.2014.6942599

  34. Rüssmeier, N., Hahn, A., Nicklas, D., Zielinski, O.: Ad-hoc situational awareness by optical sensors in a research port maritime environment, approved networking and sensor fusion technologies (2016)

    Google Scholar 

  35. Schiaretti, M., Chen, L., Negenborn, R.R.: Survey on autonomous surface vessels: part I - a new detailed definition of autonomy levels. In: Bektaş, T., Coniglio, S., Martinez-Sykora, A., Voß, S. (eds.) ICCL 2017. LNCS, vol. 10572, pp. 219–233. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68496-3_15

  36. Schubert, A.U., Kurowski, M., Gluch, M., Simanski, O Jeinsch, T.: Manoeuvring automation towards autonomous shipping. Zenodo (2018). https://doi.org/10.24868/issn.2631-8741.2018.020

  37. Snyder, F.D., Morris, D.D., Haley, P.H., Collins, R.T., Okerholm, A.M.: Autonomous river navigation. In: SPIE Optics East (2004)

    Google Scholar 

  38. Thombre, S., et al.: Sensors and AI techniques for situational awareness in autonomous ships: a review. IEEE Trans. Intell. Transp. Syst. 23(1), 64–83 (2022). https://doi.org/10.1109/TITS.2020.3023957

  39. Tian, C., Fei, L., Zheng, W., Xu, Y., Zuo, W., Lin, C.W.: Deep learning on image denoising: an overview. Neural Netw. 131, 251–275 (2020). https://doi.org/10.1016/j.neunet.2020.07.025. https://linkinghub.elsevier.com/retrieve/pii/S0893608020302665

  40. Tomar, S.: Converting video formats with FFMPEG. Linux J. 2006(146), 10 (2006)

    Google Scholar 

  41. Tonnis, M., Lindl, R., Walchshausl, L., Klinker, G.: Visualization of spatial sensor data in the context of automotive environment perception systems. In: 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 115–124 (2007). https://doi.org/10.1109/ISMAR.2007.4538835

  42. Vu, T.D., Aycard, O., Tango, F.: Object perception for intelligent vehicle applications: a multi-sensor fusion approach. In: 2014 IEEE Intelligent Vehicles Symposium Proceedings, pp. 774–780 (2014). https://doi.org/10.1109/IVS.2014.6856588

  43. Wright, R.G.: Intelligent autonomous ship navigation using multi-sensor modalities. TransNav Int. J. Marine Navig. Safety Sea Transp. 13(3), 503–510 (2019). https://doi.org/10.12716/1001.13.03.03

  44. Zhang, K., et al.: Deep image deblurring: a survey. Int. J. Comput. Vision 130(9), 2103–2130 (2022). https://doi.org/10.1007/s11263-022-01633-5. https://link.springer.com/10.1007/s11263-022-01633-5

Download references

Author information

Authors and Affiliations

  1. Institute of Information Science and Technologies, “Alessandro Faedo” (ISTI), Italian National Research Council (CNR), Via Giuseppe Moruzzi 1, 56124, Pisa, Italy

    Somnath Dutta, Fabio Ganovelli & Paolo Cignoni

Authors
  1. Somnath Dutta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabio Ganovelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paolo Cignoni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fabio Ganovelli .

Editor information

Editors and Affiliations

  1. Polytechnic Institute of Setúbal/IPS, Setúbal, Portugal

    Cédric Grueau

  2. NOVA School of Science and Technology, Caparica, Portugal

    Armanda Rodrigues

  3. Hellenic Open University, Patra, Greece

    Lemonia Ragia

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dutta, S., Ganovelli, F., Cignoni, P. (2024). Dynamic Real-Time Spatio-Temporal Acquisition and Rendering in Adverse Environments. In: Grueau, C., Rodrigues, A., Ragia, L. (eds) Geographical Information Systems Theory, Applications and Management. GISTAM 2023. Communications in Computer and Information Science, vol 2107. Springer, Cham. https://doi.org/10.1007/978-3-031-60277-1_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-60277-1_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-60276-4

  • Online ISBN: 978-3-031-60277-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation