Abstract
Attention mechanism in computer vision refers to scan, detect, and track a target object. This paper aims to develop and virtually train a machine learning model for object attention mechanism, combining object detection and mechanical automation. For this, we use Unity 3D Engine to model a simple scene in which two virtual cameras align together to realize a monocular attention in specific objects. Deep reinforcement learning, via ML-agent’s library, was used to train a model that aligns the virtual cameras. Moreover, the model was transferred to a physical camera to replicate the performance of attention mechanism.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
ml agents@unity3d.com: unity ml-agents toolkit (2022). https://github.com/Unity-Technologies/ml-agents/tree/develop/docs
Badue, C., et al.: Self-driving cars: a survey. Expert Syst. Appl. 165, 113816 (2021)
Baker, B., et al.: Emergent tool use from multi-agent autocurricula. arXiv preprint arXiv:1909.07528 (2019)
Grisetti, G., Kümmerle, R., Stachniss, C., Burgard, W.: A tutorial on graph-based slam. IEEE Intell. Transp. Syst. Mag. 2(4), 31–43 (2010)
Praeger, M., Xie, Y., Grant-Jacob, J.A., Eason, R.W., Mills, B.: Playing optical tweezers with deep reinforcement learning: in virtual, physical and augmented environments. Mach. Learn. Sci. Technol. 2(3), 035024 (2021)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)
Technologies, U.: Monobehaviour.fixedupdate(). unity documentation (2021)
Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7464–7475 (2023)
Ward, T.M., et al.: Computer vision in surgery. Surgery 169(5), 1253–1256 (2021)
Won, J., Gopinath, D., Hodgins, J.: Control strategies for physically simulated characters performing two-player competitive sports. ACM Trans. Graph. (TOG) 40(4), 1–11 (2021)
Zakka, K., Zeng, A., Lee, J., Song, S.: Form2Fit: learning shape priors for generalizable assembly from disassembly. In: Proceedings of the IEEE International Conference on Robotics and Automation (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Arizmendi, S., Paz, A., González, J., Ponce, H. (2024). Eye Control and Motion with Deep Reinforcement Learning: In Virtual and Physical Environments. In: Calvo, H., Martínez-Villaseñor, L., Ponce, H. (eds) Advances in Computational Intelligence. MICAI 2023. Lecture Notes in Computer Science(), vol 14391. Springer, Cham. https://doi.org/10.1007/978-3-031-47765-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-47765-2_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47764-5
Online ISBN: 978-3-031-47765-2
eBook Packages: Computer ScienceComputer Science (R0)