Abstract
This research study proposes a Q-learning-based metaheuristic algorithm framework for thermal design optimization of a shell-and-tube evaporator operating with different refrigerant mixtures, which is a highly complex real-world design problem and has not been investigated yet, in previous literature approaches before. The proposed method, called QL-HEUR, uses Q-learning as a high-level heuristic to iteratively guide the competitive recently emerged low-level metaheuristic algorithms. QL-HEUR is applied to 32 unconstrained optimization benchmark functions, and results are evaluated in statistical analysis. Moreover, three multidimensional constrained optimization problems will be solved. Respective solutions unravel that QL-HEUR is very effective in finding optimum solutions to constrained and unconstrained optimization problems. QL-HEUR is employed on the design optimization of a shell-and-tube heat exchanger running with different mixture pairs as a challenging real-world benchmark case. For the design case in which R134a–R1234yf (0.8:02) mixture is considered, 8.71% of the total cost is saved compared to the preliminary design of a heat exchanger operated with pure R1234yf refrigerant. For the second design case, the application of QL-HEUR results in a decrease of 8.93% for refrigerant composition R32–R134a (0.6:0.4) in comparison with the configuration running with pure R134a. It is also seen that the heat exchanger configuration running with pure R32 refrigerant yields the lowest total cost compared to the cases accomplished by varying mixture ratios of R290 and R32. It can be concluded that the optimum configuration of the heat exchanger operated with a refrigerant mixture can be conveniently employed for minimum total cost and global warming potential.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
References
Abd-Elaziz M, Yousri D, Mirjalili S (2021) A hybrid Harris hawks-moth-flame optimization algorithm including fractional-order chaos maps and evolutionary population dynamics. Adv Eng Softw 154:102973. https://doi.org/10.1016/j.advengsoft.2021.102973
Abualigah L, Diabat A, Mirjalili S, Abd Elaziz M, Gandomi AH (2021) The arithmetic optimization algorithm. comput. methods. Appl Mech Eng 376:113609. https://doi.org/10.1016/j.cma.2020.113609
Ahmadianfar I, Bozorg-Haddad O, Chu X (2020) Gradient-based optimizer: a new metaheuristic optimization algorithm. Inf Sci 540:131–159. https://doi.org/10.1016/j.ins.2020.06.037
Alatas B (2010) Chaotic harmony search algorithms. Appl Math Comput 216:2687–2699. https://doi.org/10.1016/j.amc.2010.03.114
Alirezajenad M, Enayatifar R, Motameni H, Nematzadeh H (2021) GSA-LA: gravitational search algorithm based on learning automata. J Exp Theor Artif Intell 33:109–125. https://doi.org/10.1080/0952813X.2020.1725650
Andrei N (2013) Nonlinear optimization applications using the GAMS technology. Springer, London
Blum C, Roli A (2003) Metaheuristic in combinatorial optimization: overview and conceptual comparison. ACM Comput Surv 35:268–308. https://doi.org/10.1145/937503.937505
Caputo AC, Pelagagge PM, Salini P (2008) Heat exchanger design based on economic investigation. Appl Therm Eng 28:1151–1159. https://doi.org/10.1016/j.applthermaleng.2007.08.010
Chen J, Yu J (2008) Performance of a new refrigeration cycle using refrigerant mixture R32/R134a for residential air conditioner applications. Energy Build 40:2022–2027. https://doi.org/10.1016/j.enbuild.2008.05.003
Choi TY, Kim YJ, Kim MS, Ro ST (2000) Evaporation heat transfer of R-32, R134a, R-32/134a, and R-32/125/134a inside a horizontal smooth tube. Int J Heat Mass Transf 43:3651–3660. https://doi.org/10.1016/S0017-9310(00)00005-3
Daniali OA, Toghraie D, Eftekhari SA (2020) Thermo-hydraulic and economic optimization of Iranol refinery oil heat exchanger with Copper oxide nanoparticles using MOMBO. Physica A 540:123010. https://doi.org/10.1016/j.physa.2019.123010
Esfe MH, Hajmohammad H, Toghraie D, Rostamian H, Mahian O, Wongwises S (2017) Multi-objective optimization of nanofluid flow in double tube heat exchangers for applications in energy systems. Energy 137:160–171. https://doi.org/10.1016/j.energy.2017.06.104
Fettaka S, Thibault J, Gupta Y (2013) Design of shell and tube heat exchangers using multi-objective optimization. Int J Heat Mass Transf 60:343–354. https://doi.org/10.1016/j.ijheatmasstransfer.2012.12.047
Gandomi AH, Yang XS, Talathari S, Alavi AH (2013) Metaheuristic applications in structures and infrastructures. Elsevier, London
Gandomi AH, Yang XS (2014) Chaotic Bat Algorithm. J Comput Sci 5:224–232. https://doi.org/10.1016/j.jocs.2013.10.002
Ghafoorian M, Taghizadeh N, Beigy H (2013) Automatic abstraction in reinforcement learning using ant system algorithm. AAAI Spring Symposium Series, pp 9–14
Golcuk I, Ozsoydan FB (2021) Q-learning and hyper-heuristic based algorithm recommendation for changing environments. Eng Appl Artif Intell 102:104284. https://doi.org/10.1016/j.engappai.2021.104284
Heidari AA, Mirjalili S, Faris H, Aljarah I, Mafarja M, Chen H (2019) Harris hawks optimization: algorithm and applications. Future Gener Comput Syst 97:849–872. https://doi.org/10.1016/j.future.2019.02.028
Heredia-Aricapa Y, Belman-Flores JM, Mota-Babiloni A, Serrano-Arellano J, Garcia-Pabon JJ (2020) Overview of low GWP mixtures for the replacement of HFC refrigerants: R134a, R404A, and R410A. Int J Refrig 111:113–123. https://doi.org/10.1016/j.ijrefrig.2019.11.012
Jian X, Weng Z (2020) A logistic chaotic Jaya Algorithm for parameters identification of photovoltaic cell and module models. Optik 203:164041. https://doi.org/10.1016/j.ijleo.2019.164041
Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285. https://doi.org/10.1613/jair.301
Kakac S, Liu H, Pramuanjaroenkij A (2012) Heat exchangers: selection, rating, and thermal design, 3rd edn. CRC Press, London
Khan TA, Ling SH (2021) A novel hybrid gravitational search particle swarm optimization algorithm. Eng Appl Artif Intell 102:104263. https://doi.org/10.1016/j.engappai.2021.104263
Khodabandeh E, Tograie D, Chamka A, Mashayekhi R, Akbari O, Rozati SA (2020) Energy saving with using of elliptic pillows in turbulent flow of two-phase water-silver nanofluid in a spiral heat exchanger. Int J of Numer Method H 30:2025–2049. https://doi.org/10.1108/HFF-10-2018-0594
Lara-Montano OD, Gomez-Castro FI, Gutierrez-Antonio C (2021) Comparison of the performance of different metaheuristic methods for the optimization of shell-and-tube heat exchangers. Comput Chem Eng 152:107403. https://doi.org/10.1016/j.compchemeng.2021.107403
Liu F, Zeng G (2009) Study of genetic algorithm with reinforcement learning to solve the TSP. Expert Syst Appl 36:6995–7001. https://doi.org/10.1016/j.eswa.2008.08.026
Mirjalili S, Mirjalili SM, Lewis A (2014) Grey Wolf Optimizer. Adv Eng Softw 69:46–61. https://doi.org/10.1016/j.advengsoft.2013.12.007
Mirjalili S, Lewis A (2016) The whale optimization algorithm. Adv Eng Softw 95:51–67. https://doi.org/10.1016/j.advengsoft.2016.01.008
Mizutani FT, Pessoa FLP, Queiroz EM, Hauan S, Grossman LE (2003) Mathematical programming model for heat exchanger network synthesis including detailed heat exchanger design. 1. Shell-and-tube heat exchanger design. Ind Eng Chem Res 42:4009–4018. https://doi.org/10.1021/ie020964u
Miansari M, Valipour MA, Arasteh H, Toghraie D (2020) Energy and exergy analysis and optimization of helically grooved shell and tube heat exchangers by using Taguchi experimental design. J Therm Anal Calorim 139:3151–3164. https://doi.org/10.1007/s10973-019-08653-3
Mohanty DK (2016a) Application of firefly algorithm for design optimization of a shell and tube heat exchanger from economic point of view. Int J Therm Sci 102:228–238. https://doi.org/10.1016/j.ijthermalsci.2015.12.002
Mohanty DK (2016b) Gravitational search algorithm for economic optimization of a shell and tube heat exchanger. Appl Therm Eng 107:184–193. https://doi.org/10.1016/j.applthermaleng.2016.06.133
Müller-Steinhagen H, Heck K (1986) A simple friction pressure drop correlation for two-phase flow in pipes. Chem Eng Process 20:297–308. https://doi.org/10.1016/0255-2701(86)80008-3
Pant M, Thangaraj R, Singh VP (2009) Optimization mechanical design problems using improved differential evolution algorithm. IJRTE 1:21–25
Patel VK, Savsani VJ (2015) Heat transfer search: a novel optimization algorithm. Inf Sci 324:217–246. https://doi.org/10.1016/j.ins.2015.06.044
Qu C, Gai W, Zhong M, Zhang J (2020) A novel reinforcement learning-based grey wolf optimizer algorithm for unmanned aerial vehicles (UAVs) path planning. Appl Soft Comput 89:106099. https://doi.org/10.1016/j.asoc.2020.106099
Rakshit P, Konar A, Bhowmik P, Goswami I, Das S, Jain LC, Nagar AK (2013) Realization of an adaptive memetic algorithm suing differential evolution and Q- learning: a study in multirobot path planning. IEEE Trans Syst Man Cybern Syst 43:814–831. https://doi.org/10.1109/TSMCA.2012.2226024
Rao RV, Savsani VJ, Vakharia DP (2011) Teaching-learning-based optimization: a novel method for constrained mechanical design optimization problems. Comput Aided Des 43:303–315. https://doi.org/10.1016/j.cad.2010.12.015
Rao RV, Saroj A (2017) Economic optimization of shell-and-tube heat exchanger using Jaya algorithm with maintenance consideration. Appl Therm Eng 116:473–487. https://doi.org/10.1016/j.applthermaleng.2017.01.071
Rosenhow WM, Harnett PJ (1973) Handbook of heat transfer. McGraw-Hill, New York
Ryoo HS, Sahinidis NV (1995) Global optimization of nonconvex NLPs and MINLPs with applications in process design. Comput Chem Eng 19:551–566. https://doi.org/10.1016/0098-1354(94)00097-2
Sadhu AK, Konar A (2017) Improving the speed of convergence of multi-agent Q-learning for cooperative task-planning by a robot-team. Rob Auton Syst 92:66–80. https://doi.org/10.1016/j.robot.2017.03.003
Sadhu AK, Konar A, Bhattacharjee T, Das S (2018) Synergism of firefly algorithm and Q- learning for robot arm path planning. Swarm Evol Comput 43:50–68. https://doi.org/10.1016/j.swevo.2018.03.014
Sadollah A, Sayyaadi H, Yoo DG, Lee HM, Kim JH (2018) Mine blast harmony search: a new hybrid optimization method for improving exploration and exploitation capabilities. Appl Soft Comput 68:548–564. https://doi.org/10.1016/j.asoc.2018.04.010
Samadifar M, Toghraie D (2018) Numerical simulation of heat transfer enhancement in a plate-fing heat exchanger using a new type of vortex generators. Appl Therm Eng 133:671–681. https://doi.org/10.1016/j.applthermaleng.2018.01.062
Samma H, Lim CP, Junita MS (2016) A new reinforcement learning-based memetic particle swarm optimizer. Appl Soft Comput 43:276–297. https://doi.org/10.1016/j.asoc.2016.01.006
Samma H, Mohamad-Saleh J, Suandi SA, Lahasan B (2020) Q-learning based simulated annealing algorithm for constrained engineering design problems. Neural Comput Appl 32:5147–5161. https://doi.org/10.1007/s00521-019-04008-z
Schittkowski K (1987) More test examples for nonlinear programming codes. Lecture notes on economics and mathematical systems, vol 282, Berlin, Springer
Serth RW (2007) Process heat transfer—principles and applications. Elsevier Science & Technology Books, Amsterdam
Seyyedabbasi A, Aliyev R, Kiani F, Gulle MU, Basyildiz H, Shah MA (2021) Hybrid algorithms based on combining reinforcement learning and metaheuristic methods to solve global optimization problems. Knowl Based Syst 223:107044. https://doi.org/10.1016/j.knosys.2021.107044
Sinnott RK (2005) Chemical Engineering Design: Principles, Practice and Economics of Plant and Proces Design, 5th edn. Butterworth-Heinemann, Oxford
Shah RK, Sekulic DP (2003) Fundamentals of heat exchanger design. Wiley, New York
Sutton RS, Barto AG (1998) Reinforcement Learning: an introduction. MIT Press, London
Şencan SA, Kiliç B, Kiliç U (2011) Design and economic optimization of shell and tube heat exchangers using Artificial Bee Colony (ABC) algorithm. Energy Convers Manag 52:3356–3362. https://doi.org/10.1016/j.enconman.2011.07.003
Taal M, Bulatov I, Klemes J, Stehlik P (2003) Cost estimation and energy price forecast for the economic evaluation of retrofit projects. Appl Therm Eng 23:1819–1835. https://doi.org/10.1016/S1359-4311(03)00136-4
Tian Q, Cai D, Ren L, Tang W, Xie Y, He G, Liu F (2015) An experimental investigation of refrigerant mixture R32/R290 as drop-in replacement for HFC410A in household air conditioners. Int J Refrig 57:216–228. https://doi.org/10.1016/j.ijrefrig.2015.05.005
Watkins CJCH, Dayan P (1992) Q-Learning. Mach Learn 8:279–292. https://doi.org/10.1007/BF00992698
Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. Evolut Comput, IEEE Trans 1:67–82. https://doi.org/10.1109/4235.585893
Xu Y, Pi D (2020) A reinforcement learning-based communication topology in particle swarm optimization. Neural Comput Appl 32:10007–10032. https://doi.org/10.1007/s00521-019-04527-9
Yadav S, Yadav R, Kumar A, Kumar M (2021) A novel approach for optimal design of digital FIR filter using grasshopper optimization algorithm. ISA Trans 108:196–206. https://doi.org/10.1016/j.isatra.2020.08.032
Yadav D, Verma OP (2020) Energy optimization of Multiple Stage Evaporator system using Water Cycle Algorithm. Heliyon 6:e04349. https://doi.org/10.1016/j.heliyon.2020.e04349
Yang XS (2010) Engineering optimization: an introduction with metaheuristic applications. John Wiley and Sons, NJ
Yang M, Zhang H, Meng Z, Qin Y (2019) Experimental study on R1234yf/R134a mixture (R513A) as R134a replacement domestic refrigerator. Appl Therm Eng 146:540–547. https://doi.org/10.1016/j.applthermaleng.2018.09.122
Yarmohammadi S, Mohammadzadeh K Farhadi M, Hajmiri H, Modir Alireza (2020) Multi-objective optimization of thermal and flow characteristics of R-404A evaporation through corrugated tubes. J Energy Storage 27:101137 https://doi.org/10.1016/j.est.2019.101137
Zamli KZ, Din F, Ahmed BS, Bures M (2018) A hybrid Q-learning sine-cosine-based strategy for addressing the combinatorial test suite minimization problem. PLoS ONE 13(5):e0195675. https://doi.org/10.1371/journal.pone.0195675
Zeng X, Hammid AT, Kumar NM, Subramaniam U, Almakhles DJ (2021) A grasshopper optimization algorithm for optimal short-term hydrothermal scheduling. Energy Rep 7:314–323. https://doi.org/10.1016/j.egyr.2020.12.038
Funding
This research did not receive any funding or grant from an agency or corporation.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Turgut, O.E., Turgut, M.S. & Kırtepe, E. Q-learning-based metaheuristic algorithm for thermoeconomic optimization of a shell-and-tube evaporator working with refrigerant mixtures. Soft Comput 27, 16201–16241 (2023). https://doi.org/10.1007/s00500-023-08016-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-023-08016-z