This paper presents a dynamic Q-learning (DFQL) method that is capable of tuning the fuzzy inference systems (FIS) online. On-line self-organizing learning is developed so that structure and parameters identification are accomplished automatically and simultaneously based only on Q-learning. Self-organizing fuzzy inference is introduced to calculate actions and Q-functions so as to enable us to deal with continuous-valued states and actions. Fuzzy rules provide a natural mean to incorporate the bias components for rapid reinforcement learning. Experimental results and comparative studies with the fuzzy Q-learning the wall following task of mobile robots demonstrate the superiority of the proposed DFQL method.