Skip to main content
Springer Nature Link
Log in

Temporal feature markers for event cameras

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In this paper, a marker and its real-time tracking method are proposed. Different from existing technologies that recognize the markers by spatial event features, which lead to many practical problems, we discover the possibility of using temporal features. Strobe LEDs (light emitting diode) are used as markers to produce periodically flipping events, and a fast clustering-based algorithm is designed to track and recognize these markers simultaneously. Experiments demonstrate that our methods have superior speed and accuracy compared to state-of-the-arts. The markers can be stably tracked in many challenging situations, thus can be used in various visual tracking applications. The proposed method introduces a new marker and its corresponding recognition algorithm for event camera-based targets tracking, offering a reliable solution for various applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Algorithm 1
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

All data included in this study are available upon request by contact with the corresponding author.

References

  1. Aladem, M., Rawashdeh, S.A.: A multi-cluster tracking algorithm with an event camera. In: Proceedings of the IEEE National Aerospace and Electronics Conference, pp. 391–397 (2019). https://doi.org/10.1109/NAECON46414.2019.9058204

  2. Alzugaray, I., Chli, M.: Asynchronous multi-hypothesis tracking of features with event cameras. In: Proceedings of the IEEE International Conference of 3D Vision, pp. 269–278 (2019). https://doi.org/10.1109/3DV.2019.00038

  3. Chamorro, W., Sola, J., Andrade-Cetto, J.: Event-based line slam in real-time. IEEE Robot. Automat. Lett. 7(3), 8146–8153 (2022)

    Article  Google Scholar 

  4. Dietsche, A., Cioffi, G., Hidalgo-Carrió, J., Scaramuzza, D.: Powerline tracking with event cameras. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 6990–6997 (2021). https://doi.org/10.48550/arXiv.2108.00515

  5. Gallego, G., Lund, J.E., Mueggler, E., Rebecq, H., Delbruck, T., Scaramuzza, D.: Event-based, 6-DOF camera tracking from photometric depth maps. IEEE Trans. Pattern Anal. Mach. Intell. 40(10), 2402–2412 (2017). https://doi.org/10.1109/TPAMI.2017.2769655

    Article  Google Scholar 

  6. Li, X.: Real-time digital twins end-to-end multi-branch object detection with feature level selection for healthcare. J. Real-Time Image Process. 19(5), 921–930 (2022)

    Article  MathSciNet  Google Scholar 

  7. Li, X., Yi, W., Chi, H.L., Wang, X., Chan, A.P.: A critical review of virtual and augmented reality (VR/AR) applications in construction safety. Automat. Constr. 86, 150–162 (2018)

    Article  Google Scholar 

  8. Lichtsteiner, P., Posch, C., Delbruck, T.: A \(128\times 128\) 120 db 15 \(\mu\)s latency asynchronous temporal contrast vision sensor. IEEE J. Solid-State Circuits 43(2), 566–576 (2008). https://doi.org/10.1109/JSSC.2007.914337

    Article  Google Scholar 

  9. Liu, J., Wang, Z., Xu, M.: DeepMTT: a deep learning maneuvering target-tracking algorithm based on bidirectional LSTM network. Inf. Fusion 53, 289–304 (2020)

    Article  Google Scholar 

  10. Loch, A., Haessig, G., Vincze, M.: Event-based high-speed low-latency fiducial marker tracking (2021). arXiv preprint. https://doi.org/10.48550/arXiv.2110.05819

  11. Messikommer, N., Fang, C., Gehrig, M., Scaramuzza, D.: Data-driven feature tracking for event cameras, pp. 5642–5651. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, IEEE (2023)

    Google Scholar 

  12. Munda, G., Reinbacher, C., Pock, T.: Real-time intensity-image reconstruction for event cameras using manifold regularisation. Int. J. Comput. Vis. 126(12), 1381–1393 (2018). https://doi.org/10.5244/C.30.9

    Article  Google Scholar 

  13. Munoz-Salinas, R., Medina-Carnicer, R.: UcoSLAM: simultaneous localization and mapping by fusion of keypoints and squared planar markers. Pattern Recognit. 101, 107193 (2020)

    Article  Google Scholar 

  14. Müggler, E., Bartolozzi, C., Scaramuzza, D.: Fast event-based corner detection. In: British Machine Vision Conference, pp. 1–12 (2017). https://doi.org/10.5244/C.31.33

  15. Ong, S.K., Yew, A., Thanigaivel, N.K., Nee, A.Y.: Augmented reality-assisted robot programming system for industrial applications. Robot. Comput. Integr. Manuf. 61, 101820 (2020)

    Article  Google Scholar 

  16. Ran, T., Yuan, L., Zhang, J.: Scene perception based visual navigation of mobile robot in indoor environment. ISA Trans. 109, 389–400 (2021)

    Article  Google Scholar 

  17. Rebecq, H., Horstschäfer, T., Gallego, G., Scaramuzza, D.: EVO: a geometric approach to event-based 6-DOF parallel tracking and mapping in real time. IEEE Robot. Autom. Lett. 2(2), 593–600 (2016). https://doi.org/10.1109/LRA.2016.2645143

    Article  Google Scholar 

  18. Rodríguez-Gómez, J.P., Eguíluz, A.G., Martínez-de Dios, J., Ollero, A.: Asynchronous event-based clustering and tracking for intrusion monitoring in UAS. In: Proceedings of IEEE International Conference Robotics and Automation, pp. 8518–8524 (2020). https://doi.org/10.1109/ICRA40945.2020.9197341

  19. Shorfuzzaman, M., Hossain, M.S., Alhamid, M.F.: Towards the sustainable development of smart cities through mass video surveillance: a response to the COVID-19 pandemic. Sustain. Cities Soc. 64, 102582 (2021)

    Article  Google Scholar 

  20. Wang, X., Li, J., Zhu, L., Zhang, Z., Chen, Z., Li, X., Wang, Y., Tian, Y., Wu, F.: VisEvent: reliable object tracking via collaboration of frame and event flows. IEEE Trans. Cybern. 54(3), 997–2010 (2023)

    Google Scholar 

  21. Wang, Y., Sun, Q., Liu, Z., Gu, L.: Visual detection and tracking algorithms for minimally invasive surgical instruments: a comprehensive review of the state-of-the-art. Robot. Auton. Syst. 149, 103945 (2022)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (No. 62003007), the Nature Science Foundation of Fujian Province (No. 2022J05111), the University-Industry Cooperation Project in Fujian Province (No. 2022Y4001), and the University-Industry Cooperation Project in Fujian Province (No. 2022I0026).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering and Automation, Fuzhou University, Xue Yuan Road No.2, Fuzhou, Fujian, China

    Yue You, Mingzhu Zhu & Bingwei He

  2. The Shengli Clinical Medical College of Fujian Medical University, 134 East Street, Fuzhou, Fujian, China

    Yihong Wang

Authors
  1. Yue You
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Mingzhu Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Bingwei He
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Yihong Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

YY performed tracking landmark experiments based on event-camera and completed contrast with state-of-the-arts. MZ is a major contributor to writing the manuscript. BH conceived and designed of the study. YW analysed most of the data. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Mingzhu Zhu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

You, Y., Zhu, M., He, B. et al. Temporal feature markers for event cameras. J Real-Time Image Proc 21, 41 (2024). https://doi.org/10.1007/s11554-024-01422-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-024-01422-y

Keywords

Mathematics Subject Classification