Abstract
The DC bus voltage of AC/DC converters is conventionally regulated by proportional plus integral (PI)-based controllers. However, such controllers can’t provide good performance over the entire operating region and their performance is also affected by the change of parameters of passive components depending on temperature during the converters’ operation. On the other hand, reinforcement learning (RL), which is one of the machine learning methods, can become immune to parameter changes as it maintains training during the converter operation. In this paper, an RL-based control algorithm is proposed for an AC/DC dual active bridge (DAB) converter which operates with improved hybrid current modulation (iHCM). In the proposed method, the model-free Q-learning algorithm of RL is used to train an agent to regulate the DC bus voltage. The proposed algorithm is verified for various load and disturbance conditions by MATLAB/Simulink simulations, and it is compared with a PI controller which is tuned with Internal Model Control (IMC) method. According to the simulation results, besides the online learning advantage, the proposed method creates a small settling time and overshoot at the start-up for light load conditions, unlike the PI controller. On the other hand, during the change of dynamic load and AC grid voltage, it creates smaller voltage oscillations in the output DC voltage and regulates it faster. Furthermore, since the proposed method keeps the duty cycle value constant in each grid period, it produces lower total harmonic distortion (THD) than the PI controller.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig7_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig8_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig9_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig10_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig11_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig12_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig13_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig14_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig15_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig16_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig17_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig18_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig19_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig20_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig21_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs00521-021-06698-w/MediaObjects/521_2021_6698_Fig22_HTML.png)
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Kroics K, Husev O, Tytelmaier K et al (2018) An overview of bidirectional AC-DC grid connected converter topologies for low voltage battery integration. Int J Power Electron Drive Syst 9(3):1223. https://doi.org/10.11591/ijpeds.v9.i3.pp1223-1239
Singh B, Singh BN, Chandra A et al (2003) A review of single-phase improved power quality AC-DC converters. IEEE Trans Ind Electron 50:962–981. https://doi.org/10.1109/TIE.2003.817609
Singh B, Singh S, Chandra A, Al-Haddad K (2011) Comprehensive study of single-phase AC-DC power factor corrected converters with high-frequency isolation. IEEE Trans Ind Informatics 7:540–556. https://doi.org/10.1109/TII.2011.2166798
Kheraluwala MH, Donker RWD (1993) Single phase unity power factor control for dual active bridge converter. In: Industry applications society annual meeting. pp 909–916
Everts J, Krismer F, Van Den Keybus J et al (2014) Optimal ZVS modulation of single-phase single-stage bidirectional DAB AC-DC converters. IEEE Trans Power Electron 29:3954–3970. https://doi.org/10.1109/TPEL.2013.2292026
Jauch F, Biela J (2012) Single-phase single-stage bidirectional isolated ZVS AC-DC converter with PFC. In: 15th International power electronics and motion control conference and exposition, EPE-PEMC 2012 ECCE Europe
Jauch F, Biela J (2016) Combined phase-shift and frequency modulation of a dual-active-bridge AC-DC converter with PFC. IEEE Trans Power Electron 31:8387–8397. https://doi.org/10.1109/TPEL.2016.2515850
Tian Q, Huang AQ, Bai H, et al (2016) A novel light load performance enhanced variable-switching-frequency and hybrid single-dual-phase-shift control for single-stage dual-active-bridge based AC/DC converter. IECON Proc (Industrial Electron Conf 1227–1232. https://doi.org/10.1109/IECON.2016.7793438
Taylor A, Liu G, Bai H et al (2018) Multiple-phase-shift control for a dual active bridge to secure zero-voltage switching and enhance light-load performance. IEEE Tran Power 33:4584–4588
Weise ND, Castelino G, Basu K, Mohan N (2014) A single-stage dual-active-bridge-based soft switched AC – DC converter With open-loop power factor correction and other advanced features. IEEE Tran Power Electron 29:4007–4016
Zengin S, Boztepe M (2020) A novel current modulation method to eliminate low-frequency harmonics in single-stage dual active bridge AC-DC converter. IEEE Trans Ind Electron 67:1048–1058. https://doi.org/10.1109/TIE.2019.2898597
Huang J, Wang Y, Li Z, Lei W (2016) Unified triple-phase-shift control to minimize current stress and achieve full soft-switching of isolated bidirectional DC – DC converter. IEEE Tran Ind Electron 63:4169–4179
Calderon C, Barrado A, Rodriguez A, et al (2017) Dual active bridge (TPS-DAB) with soft switching in the whole output power range. In: 11th IEEE International conference on compatibility, power electron. and power Eng. IEEE, pp 217–222
Tamyurek B, Kirimer B (2015) An interleaved high-power flyback inverter for photovoltaic applications. IEEE Trans Power Electron 30:3228–3241. https://doi.org/10.1109/TPEL.2014.2332503
Guo X, Sha D, Zhang J, Liao X (2017) Unified boundary trapezoidal modulation control utilizing fixed duty cycle compensation and magnetizing current design for dual active bridge DC–DC converter. IEEE Tran Power Electron 32:2243–2252. https://doi.org/10.1109/TPEL.2016.2555328
Zhang J, Sha D, Ma P (2021) A dual active bridge DC-DC based single stage AC-DC converter with seamless mode transition and high power factor. IEEE Trans Ind Electron in press: https://doi.org/10.1109/TIE.2021.3057016
Erickson RW, Maksimovic D (2001) Fundamentals of power electronics, Springer US, Boston, MA https://doi.org/10.1007/b100747
Xue D, Chen Y, Atherton D (2008) Linear feedback control: analysis and design with MATLAB. Soc Ind Math 10(1137/1):9780898718621
Garcia CE, Morari M (1982) Internal model control. a unifying review and some new results. Ind Eng Chem Process Des Dev 21:308–323. https://doi.org/10.1021/i200017a016
Chew IM, Wong F, Bono A, et al (2017) Improved cascade control tuning for temperature control system. In: International conference on computational science and technology. pp 410–419
Chung HSH, Wang H, Blaabjerg F, Pecht M (2016) Reliability of power electronic converter systems
Dehbi A, Wondrak W, Ousten Y, Danto Y (2002) High temperature reliability testing of aluminum and tantalum electrolytic capacitors. Microelectron Reliab 42:835–840. https://doi.org/10.1016/S0026-2714(02)00021-5
Colmenares J, Kargarrazi S, Elahipanah H, et al (2016) High-temperature passive components for extreme environments. In: WiPDA 2016 - 4th IEEE Workshop on wide bandgap power devices and applications. pp 271–274
Rathi KJ, Ali MS (2017) Neural network controller for power electronics circuits. Int J Artif Intelligience 6:49–56
Lin B-R (1995) Power converter control based on neural and fuzzy methods. Electr Power Syst Res 35:193–206
Chettibi N, Mellit A, Sulligoi G, Pavan AM (2018) Adaptive neural network-based control of a hybrid AC / DC microgrid. IEEE Trans Smart Grid 9:1667–1679
Phan BC, Lai Y, Lin CE (2020) A deep reinforcement learning-based MPPT control for PV systems under partial shading condition. Sensors 20:1–23
Bag A, Subudhi B, Ray PK (2019) A combined reinforcement learning and sliding mode control Scheme for Grid Integration of a PV system. CSEE J Power Energy Sytems 5:498–506. https://doi.org/10.17775/CSEEJPES.2017.01000
Kofinas P, Doltsinis S, Dounis AI, Vouros GA (2017) A reinforcement learning approach for MPPT control method of photovoltaic sources. Renew Energy 108:461–473. https://doi.org/10.1016/j.renene.2017.03.008
Tang Y, Hu W, Xiao J et al (2020) Reinforcement learning based efficiency optimization scheme for the DAB DC-DC converter with triple-phase-shift modulation. IEEE Trans Ind Electron 68:7350–7361. https://doi.org/10.1109/tie.2020.3007113
Zengin S, Boztepe M (2021) Improved hybrid current modulation for bidirectional power flow of single - stage dual active bridge AC / DC converter. Electr Eng in press: https://doi.org/10.1007/s00202-021-01229-1
Tokic M, Palm G (2011) Value-difference based exploration: Adaptive control between epsilon-greedy and softmax. In: Advances in artificial intelligence. pp 335–346
Mnih V, Kavukcuoglu K, Silver D et al (2015) Human-level control through deep reinforcement learning. Nature 518:529–533. https://doi.org/10.1038/nature14236
Reinforcement-learning Toolbox. In: Matlab. https://www.mathworks.com/help/reinforcement-learning/ref/rlqagent.html. Accessed 28 May 2021
Morales EF, Zaragoza JH (2014) An introduction to reinforcement learning
First Order Plus Dead Time (FOPDT). In: Dyn. Control. https://apmonitor.com/pdc/index.php/Main/FirstOrderSystems. Accessed 1 Jun 2021
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author declares that I have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zengin, S. Reinforcement learning-based control of improved hybrid current modulated dual active bridge AC/DC converter. Neural Comput & Applic 34, 5417–5430 (2022). https://doi.org/10.1007/s00521-021-06698-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-021-06698-w