Skip to main content

Advertisement

SpringerLink
Log in

Reinforcement learning-based control of improved hybrid current modulated dual active bridge AC/DC converter

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The DC bus voltage of AC/DC converters is conventionally regulated by proportional plus integral (PI)-based controllers. However, such controllers can’t provide good performance over the entire operating region and their performance is also affected by the change of parameters of passive components depending on temperature during the converters’ operation. On the other hand, reinforcement learning (RL), which is one of the machine learning methods, can become immune to parameter changes as it maintains training during the converter operation. In this paper, an RL-based control algorithm is proposed for an AC/DC dual active bridge (DAB) converter which operates with improved hybrid current modulation (iHCM). In the proposed method, the model-free Q-learning algorithm of RL is used to train an agent to regulate the DC bus voltage. The proposed algorithm is verified for various load and disturbance conditions by MATLAB/Simulink simulations, and it is compared with a PI controller which is tuned with Internal Model Control (IMC) method. According to the simulation results, besides the online learning advantage, the proposed method creates a small settling time and overshoot at the start-up for light load conditions, unlike the PI controller. On the other hand, during the change of dynamic load and AC grid voltage, it creates smaller voltage oscillations in the output DC voltage and regulates it faster. Furthermore, since the proposed method keeps the duty cycle value constant in each grid period, it produces lower total harmonic distortion (THD) than the PI controller.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Similar content being viewed by others

References

  1. Kroics K, Husev O, Tytelmaier K et al (2018) An overview of bidirectional AC-DC grid connected converter topologies for low voltage battery integration. Int J Power Electron Drive Syst 9(3):1223. https://doi.org/10.11591/ijpeds.v9.i3.pp1223-1239

    Article  Google Scholar 

  2. Singh B, Singh BN, Chandra A et al (2003) A review of single-phase improved power quality AC-DC converters. IEEE Trans Ind Electron 50:962–981. https://doi.org/10.1109/TIE.2003.817609

    Article  Google Scholar 

  3. Singh B, Singh S, Chandra A, Al-Haddad K (2011) Comprehensive study of single-phase AC-DC power factor corrected converters with high-frequency isolation. IEEE Trans Ind Informatics 7:540–556. https://doi.org/10.1109/TII.2011.2166798

    Article  Google Scholar 

  4. Kheraluwala MH, Donker RWD (1993) Single phase unity power factor control for dual active bridge converter. In: Industry applications society annual meeting. pp 909–916

  5. Everts J, Krismer F, Van Den Keybus J et al (2014) Optimal ZVS modulation of single-phase single-stage bidirectional DAB AC-DC converters. IEEE Trans Power Electron 29:3954–3970. https://doi.org/10.1109/TPEL.2013.2292026

    Article  Google Scholar 

  6. Jauch F, Biela J (2012) Single-phase single-stage bidirectional isolated ZVS AC-DC converter with PFC. In: 15th International power electronics and motion control conference and exposition, EPE-PEMC 2012 ECCE Europe

  7. Jauch F, Biela J (2016) Combined phase-shift and frequency modulation of a dual-active-bridge AC-DC converter with PFC. IEEE Trans Power Electron 31:8387–8397. https://doi.org/10.1109/TPEL.2016.2515850

    Article  Google Scholar 

  8. Tian Q, Huang AQ, Bai H, et al (2016) A novel light load performance enhanced variable-switching-frequency and hybrid single-dual-phase-shift control for single-stage dual-active-bridge based AC/DC converter. IECON Proc (Industrial Electron Conf 1227–1232. https://doi.org/10.1109/IECON.2016.7793438

  9. Taylor A, Liu G, Bai H et al (2018) Multiple-phase-shift control for a dual active bridge to secure zero-voltage switching and enhance light-load performance. IEEE Tran Power 33:4584–4588

    Article  Google Scholar 

  10. Weise ND, Castelino G, Basu K, Mohan N (2014) A single-stage dual-active-bridge-based soft switched AC – DC converter With open-loop power factor correction and other advanced features. IEEE Tran Power Electron 29:4007–4016

    Article  Google Scholar 

  11. Zengin S, Boztepe M (2020) A novel current modulation method to eliminate low-frequency harmonics in single-stage dual active bridge AC-DC converter. IEEE Trans Ind Electron 67:1048–1058. https://doi.org/10.1109/TIE.2019.2898597

    Article  Google Scholar 

  12. Huang J, Wang Y, Li Z, Lei W (2016) Unified triple-phase-shift control to minimize current stress and achieve full soft-switching of isolated bidirectional DC – DC converter. IEEE Tran Ind Electron 63:4169–4179

    Article  Google Scholar 

  13. Calderon C, Barrado A, Rodriguez A, et al (2017) Dual active bridge (TPS-DAB) with soft switching in the whole output power range. In: 11th IEEE International conference on compatibility, power electron. and power Eng. IEEE, pp 217–222

  14. Tamyurek B, Kirimer B (2015) An interleaved high-power flyback inverter for photovoltaic applications. IEEE Trans Power Electron 30:3228–3241. https://doi.org/10.1109/TPEL.2014.2332503

    Article  Google Scholar 

  15. Guo X, Sha D, Zhang J, Liao X (2017) Unified boundary trapezoidal modulation control utilizing fixed duty cycle compensation and magnetizing current design for dual active bridge DC–DC converter. IEEE Tran Power Electron 32:2243–2252. https://doi.org/10.1109/TPEL.2016.2555328

    Article  Google Scholar 

  16. Zhang J, Sha D, Ma P (2021) A dual active bridge DC-DC based single stage AC-DC converter with seamless mode transition and high power factor. IEEE Trans Ind Electron in press: https://doi.org/10.1109/TIE.2021.3057016

    Article  Google Scholar 

  17. Erickson RW, Maksimovic D (2001) Fundamentals of power electronics, Springer US, Boston, MA https://doi.org/10.1007/b100747

  18. Xue D, Chen Y, Atherton D (2008) Linear feedback control: analysis and design with MATLAB. Soc Ind Math 10(1137/1):9780898718621

    Google Scholar 

  19. Garcia CE, Morari M (1982) Internal model control. a unifying review and some new results. Ind Eng Chem Process Des Dev 21:308–323. https://doi.org/10.1021/i200017a016

    Article  Google Scholar 

  20. Chew IM, Wong F, Bono A, et al (2017) Improved cascade control tuning for temperature control system. In: International conference on computational science and technology. pp 410–419

  21. Chung HSH, Wang H, Blaabjerg F, Pecht M (2016) Reliability of power electronic converter systems

  22. Dehbi A, Wondrak W, Ousten Y, Danto Y (2002) High temperature reliability testing of aluminum and tantalum electrolytic capacitors. Microelectron Reliab 42:835–840. https://doi.org/10.1016/S0026-2714(02)00021-5

    Article  Google Scholar 

  23. Colmenares J, Kargarrazi S, Elahipanah H, et al (2016) High-temperature passive components for extreme environments. In: WiPDA 2016 - 4th IEEE Workshop on wide bandgap power devices and applications. pp 271–274

  24. Rathi KJ, Ali MS (2017) Neural network controller for power electronics circuits. Int J Artif Intelligience 6:49–56

    Google Scholar 

  25. Lin B-R (1995) Power converter control based on neural and fuzzy methods. Electr Power Syst Res 35:193–206

    Article  Google Scholar 

  26. Chettibi N, Mellit A, Sulligoi G, Pavan AM (2018) Adaptive neural network-based control of a hybrid AC / DC microgrid. IEEE Trans Smart Grid 9:1667–1679

    Google Scholar 

  27. Phan BC, Lai Y, Lin CE (2020) A deep reinforcement learning-based MPPT control for PV systems under partial shading condition. Sensors 20:1–23

    Article  Google Scholar 

  28. Bag A, Subudhi B, Ray PK (2019) A combined reinforcement learning and sliding mode control Scheme for Grid Integration of a PV system. CSEE J Power Energy Sytems 5:498–506. https://doi.org/10.17775/CSEEJPES.2017.01000

    Article  Google Scholar 

  29. Kofinas P, Doltsinis S, Dounis AI, Vouros GA (2017) A reinforcement learning approach for MPPT control method of photovoltaic sources. Renew Energy 108:461–473. https://doi.org/10.1016/j.renene.2017.03.008

    Article  Google Scholar 

  30. Tang Y, Hu W, Xiao J et al (2020) Reinforcement learning based efficiency optimization scheme for the DAB DC-DC converter with triple-phase-shift modulation. IEEE Trans Ind Electron 68:7350–7361. https://doi.org/10.1109/tie.2020.3007113

    Article  Google Scholar 

  31. Zengin S, Boztepe M (2021) Improved hybrid current modulation for bidirectional power flow of single - stage dual active bridge AC / DC converter. Electr Eng in press: https://doi.org/10.1007/s00202-021-01229-1

    Article  Google Scholar 

  32. Tokic M, Palm G (2011) Value-difference based exploration: Adaptive control between epsilon-greedy and softmax. In: Advances in artificial intelligence. pp 335–346

  33. Mnih V, Kavukcuoglu K, Silver D et al (2015) Human-level control through deep reinforcement learning. Nature 518:529–533. https://doi.org/10.1038/nature14236

    Article  Google Scholar 

  34. Reinforcement-learning Toolbox. In: Matlab. https://www.mathworks.com/help/reinforcement-learning/ref/rlqagent.html. Accessed 28 May 2021

  35. Morales EF, Zaragoza JH (2014) An introduction to reinforcement learning

  36. First Order Plus Dead Time (FOPDT). In: Dyn. Control. https://apmonitor.com/pdc/index.php/Main/FirstOrderSystems. Accessed 1 Jun 2021

Download references

Author information

Authors and Affiliations

  1. Electrical and Electronics Engineering Department, Ege University, Bornova, Izmir, Turkey

    Sinan Zengin

Authors
  1. Sinan Zengin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sinan Zengin.

Ethics declarations

Conflict of interest

The author declares that I have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zengin, S. Reinforcement learning-based control of improved hybrid current modulated dual active bridge AC/DC converter. Neural Comput & Applic 34, 5417–5430 (2022). https://doi.org/10.1007/s00521-021-06698-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06698-w

Keywords

Search

Navigation