Abstract
Sheet metal manufacturing is finding wide applications in automotive and aerospace industries. Handling of giant sheet materials in manufacturing industries is one of the key problems. Utilization of robots, viz SwarmItFIX, will address this problem and automate the fixturing process, which greatly reduces lead time and thus the production cost. Implementation of intelligence into the robots will further improve efficiency in handling and reduce manufacturing inaccuracies. In this work, two different novel planners are proposed which do path planning for the heads of the SwarmItFIX agents. The environment of the problem is modeled as a Markov Decision Problem. The first planner uses the Value Iteration and Policy Iteration (PI) algorithms individually and the second planner performs the Monte Carlo control reinforcement learning. Finally, when the simulation is done and parameters of the proposed three algorithms along with existing Constraint Satisfaction Problem algorithm are compared with each other. It is observed that the proposed PI algorithm returns the plan much faster than the other algorithms. In the near future, the efficient planning model will be tested and implemented into the SwarmItFIX setup at the PMAR laboratory, University of Genoa, Italy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bi, Z. M., & Zhang, W. J. (2001). Flexible fixture design and automation: Review, issues, and future directions. International Journal of Production Research,39(13), 2867–2894. https://doi.org/10.1080/00207540110054579.
Chen, F., Sekiyama, K., Huang, J., Sun, B., Sasaki, H., & Fukuda, T. (2011). An assembly strategy scheduling method for human and robot coordinated cell manufacturing. International Journal of Intelligent Computing and Cybernetics,4(4), 487–510. https://doi.org/10.1108/17563781111186761.
Daoud, S., Chehade, H., Yalaoui, F., & Amodeo, L. (2014). Efficient metaheuristics for pick and place robotic systems optimization. Journal of Intelligent Manufacturing,25(1), 27–41. https://doi.org/10.1007/s10845-012-0668-z.
De Leonardo, L., Zlatanov, D., Zoppi, M., & Molfino, R. (2013). Design of the locomotion and docking system of the SwarmItFIX mobile fixture agent. Procedia Engineering,64, 1416–1425. https://doi.org/10.1016/j.proeng.2013.09.223.
Driessens, K. (2004). Thesis: Relational reinforcement learning. AI Communications,18, 71–73.
Girard, J., & Reza Emami, M. (2015). Concurrent Markov decision processes for robot team learning. Engineering Applications of Artificial Intelligence,39, 223–234. https://doi.org/10.1016/j.engappai.2014.12.007.
Jonsson, M., & Ossbahr, G. (2010). Aspects of reconfigurable and flexible fixtures. Production Engineering,4(4), 333–339. https://doi.org/10.1007/s11740-010-0256-z.
Kasprzak, W., Szynkiewicz, W., Zlatanov, D., & Zielińska, T. (2014). A hierarchical CSP search for path planning of cooperating self-reconfigurable mobile fixtures. Engineering Applications of Artificial Intelligence,34, 85–98. https://doi.org/10.1016/j.engappai.2014.05.013.
Kasprzak, W., Zlatanov, D., Szynkiewicz, W., & Zielinńska, T. (2013). Task planning for cooperating self-reconfigurable mobile fixtures. International Journal of Advanced Manufacturing Technology,69(9–12), 2555–2568. https://doi.org/10.1007/s00170-013-5185-5.
Kuremoto, T., Obayashi, M., & Kobayashi, K. (2009). Adaptive swarm behavior acquisition by a neuro-fuzzy system and reinforcement learning algorithm. International Journal of Intelligent Computing and Cybernetics,2(4), 724–744. https://doi.org/10.1108/17563780911005854.
Leottau, D. L., Ruiz-del-Solar, J., & Babuška, R. (2018). Decentralized Reinforcement Learning of robot behaviors. Artificial Intelligence,256, 130–159. https://doi.org/10.1016/j.artint.2017.12.001.
Littman, M. L., Dean, T. L., & Kaelbling, L. P. (2013). On the complexity of solving Markov decision problems. http://arxiv.org/abs/1302.4971.
Maoudj, A., Bouzouia, B., Hentout, A., Kouider, A., & Toumi, R. (2017). Distributed multi-agent scheduling and control system for robotic flexible assembly cells. Journal of Intelligent Manufacturing,30(4), 1–16. https://doi.org/10.1007/s10845-017-1345-z.
Miljković, Z., Mitić, M., Lazarević, M., & Babić, B. (2013). Neural network Reinforcement Learning for visual control of robot manipulators. Expert Systems with Applications,40(5), 1721–1736. https://doi.org/10.1016/j.eswa.2012.09.010.
Molfino, R., Zoppi, M., & Zlatanov, D. (2009). Reconfigurable swarm fixtures. In Reconfigurable mechanisms and robots (July), pp. 730–735.
Molfino, R. M., Zoppi, M., & Zlatanov, D. (2013). Bench and method for the support and manufacturing of parts with complex geometry. United States Patent.
Mundhenk, M., Goldsmith, J., Lusena, C., & Allender, E. (2002). Complexity of finite-horizon Markov decision process problems. Journal of the ACM,47(4), 681–720. https://doi.org/10.1145/347476.347480.
Russel, S. J., & Norvig, P. (2015). Artificial intelligence a modern approiach (3rd ed.). London: Pearson Education Limited.
Sagar, K., de Leonardo, L., Molfino, R., Zielińska, T., Zieliński, C., Zlatanov, D., et al. (2017a). The SwarmItFix pilot. Procedia Manufacturing,11, 413–422. https://doi.org/10.1016/j.promfg.2017.07.128.
Sagar, K., Lugo, J. H., Molfino, R., Zlatanov, D., Zoppi, M., & Muthuswamy, S. (2018a). Coordinated selection and timing of multiple trajectories of discretely mobile robots. Procedia Computer Science,133, 676–683. https://doi.org/10.1016/j.procs.2018.07.103.
Sagar, K., Nattero, C., Muthuswamy, S., Zoppi, M., & Zlatanov, D. (2018b). Orientation planning for multi-agents with discrete-step locomotion and multiple goals. ASME. In: International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Volume 5A: 42nd Mechanisms and Robotics Conference, V05AT07A077. https://doi.org/10.1115/detc2018-86147.
Sagar, K., Zlatanov, D., Zoppi, M., Nattero, C., & Muthuswamy, S. (2017b). Multi-goal path planning for robotic agents with discrete-step locomotion. ASME. In: International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Volume 5A: 41st Mechanisms and Robotics Conference, V05AT08A033. https://doi.org/10.1115/detc2017-68011.
Sahin, C., Demirtas, M., Erol, R., Baykasoğlu, A., & Kaplanoğlu, V. (2017). A multi-agent based approach to dynamic scheduling with flexible processing capabilities. Journal of Intelligent Manufacturing,28(8), 1827–1845. https://doi.org/10.1007/s10845-015-1069-x.
Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning, an introduction (2nd ed.). Cambridge: The MIT Press.
Tohidi, H., & Algeddawy, T. (2016). Planning of modular fixtures in a robotic assembly system. Procedia CIRP,41, 252–257. https://doi.org/10.1016/j.procir.2015.12.090.
Trinh, C., Zlatanov, D., & Zoppi, M. (2016). Direct Kinematics of the Exechon Tripod (August), pp. 1–10. https://doi.org/10.1115/detc2016-60038.
Wang, Y., & De Silva, C. W. (2010). Sequential Q-learning with Kalman filtering for multi-robot cooperative transportation. IEEE/ASME Transactions on Mechatronics,15(2), 261–268. https://doi.org/10.1109/TMECH.2009.2024681.
Yang, H., Krut, S., Baradat, C., & Pierrot, F. (2011). Locomotion approach of REMORA: A reconfigurable mobile robot for manufacturing applications. In IEEE international conference on intelligent robots and systems (September), pp. 5067–5072. https://doi.org/10.1109/iros.2011.6048538.
Zecca, G., Couderc, P., Banâtre, M., & Beraldi, R. (2009). A swarm of robots using RFID tags for synchronization and cooperation. International Journal of Intelligent Computing and Cybernetics,2(4), 846–869. https://doi.org/10.1108/17563780911005890.
Zielinska, T., Kasprzak, W., Szynkiewicz, W., & Zieliński, C. (2014). Path planning for robotized mobile supports. Mechanism and Machine Theory,78, 51–64. https://doi.org/10.1016/j.mechmachtheory.2014.03.004.
Zieliński, C., Kasprzak, W., Kornuta, T., Szynkiewicz, W., Trojanek, P., Walȩcki, M., et al. (2013). Control and programming of a multi-robot-based reconfigurable fixture. Industrial Robot,40(4), 329–336. https://doi.org/10.1108/01439911311320831.
Zieliñski, C., & Winiarski, T. (2010). Motion generation in the MRROC++ robot programming framework. International Journal of Robotics Research,29(4), 386–413. https://doi.org/10.1177/0278364909348761.
Zlatanov, D., Zoppi, M., Sagar, K., Sreekumar, M., & Nattero, C. (2016). Path planning of a material handling agent with novel locomotion (August), V004T05A007. https://doi.org/10.1115/detc2016-60054.
Acknowledgements
This work is an outcome of the sponsored project (Project No.: MEC/DIMEUG-ITALY/2015-16/MSK/008/02) with financial Grant of the University of Genoa, Italy.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Veeramani, S., Muthuswamy, S., Sagar, K. et al. Artificial intelligence planners for multi-head path planning of SwarmItFIX agents. J Intell Manuf 31, 815–832 (2020). https://doi.org/10.1007/s10845-019-01479-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-019-01479-8