Improving learning ability of learning automata using chaos theory

Abstract

A learning automaton (LA) can be considered as an abstract system with a finite set of actions. LA operates by choosing an action from the set of its actions and applying it to the stochastic environment. The environment evaluates the chosen action, and automaton uses the response of the environment to update its decision-making method for selecting the next action. This process is repeated until the optimal action is found. The learning algorithm (learning scheme) determines how to use the environment response for updating the decision-making method to select the next action. In this paper, the chaos theory is incorporated with the LA and a new type of LA, namely chaotic LA (cLA), is introduced. In cLA, the chaotic numbers are used instead of the random numbers when choosing the action. The experiment results show that in most cases, the use of chaotic numbers leads to a significant improvement in the learning ability of the LA. Among the chaotic maps investigated in this paper, the Tent map has better performance than the other maps. The convergence rate/convergence time of the LA will increase/decrease by 91.4%/29.6% to 264.4%/69.1%, on average, by using the Tent map. Furthermore, the chaotic LA has more scalability than the standard LA, and its performance will not decrease significantly by increasing the problem size (number of actions).

Appendix 1

In this appendix, the results of Tables 2 and 3 are presented as a chart for further analysis. In the charts of this appendix, the red horizontal dashed line indicates the convergence rate/convergence time of the standard LA, and the bars indicate the convergence rate/convergence time of the chaotic LAs. In the blue charts (Figs. 13, 14, 15, 16, 17), which illustrate the convergence rate, chaotic LAs whose bar are over the red horizontal dashed line have better performance than the standard LA. In the green charts (Figs. 18, 19, 20, 21, 22), which illustrate the convergence time, chaotic LAs whose bar are under the red horizontal dashed line have better performance than the standard LA. By investigating these charts, we can simply conclude that the Tent chaotic LA has better performance than the other chaotic LAs and standard LA in terms of both mentioned criteria, i.e., convergence rate and convergence time.

Fig. 13

Convergence rate in the problem $P_{\text{I}}$ for different modes of action selection

Fig. 14

Convergence rate in the problem $P_{\text{II}}$ for different modes of action selection

Fig. 15

Convergence rate in the problem $P_{\text{III}}$ for different modes of action selection

Fig. 16

Convergence rate in the problem $P_{\text{IV}}$ for different modes of action selection

Fig. 17

Convergence rate in the problem $P_{\text{V}}$ for different modes of action selection

Fig. 18

Convergence time in the problem $P_{\text{I}}$ for different modes of action selection

Fig. 19

Convergence time in the problem $P_{\text{II}}$ for different modes of action selection

Fig. 20

Convergence time in the problem $P_{\text{III}}$ for different modes of action selection

Fig. 21

Convergence time in the problem $P_{\text{IV}}$ for different modes of action selection

Fig. 22

Convergence time in the problem $P_{\text{V}}$ for different modes of action selection