A Reinforcement Learning with Condition Reduced Fuzz Rules

Abstract

This paper proposes a new Q-learning method for the case where the states (conditions) and actions of systems are assumed to be continuous. The components of Q-tables are interpolated by fuzzy inference. The initial set of fuzzy rules is made of all the combinations of conditions and actions relevant to the problem. Each rule is then associated with a value by which the Q-value of a condition/action pair is estimated. The values are revised by the Q-learning algorithm so as to make the fuzzy rule system effective. Although this framework may require a huge number of the initial fuzzy rules, we will show that considerable reduction can be done by using what we call “Condition Reduced Fuzzy Rules (CRFR)”. The antecedent part of CRFR consists of all the actions and the selected conditions, and its consequent is set to be its Q-value. Finally, experimental results show that controllers with CRFRs perform equivalently to the system with the most detailed fuzzy control rules, while the total number of parameters that have to be revised through the whole learning process is reduced and the number of the revised parameters at each step of learning is increased.