This paper investigates four methods of implementing a Q-Learning Fuzzy Inference System (QFIS) algorithm to autonomously tune the parameters of a fuzzy inference system. We use an actor-critique structure and we simulate mobile robots playing the differential form of the pursuit evasion game. Both the critique and the actor are fuzzy inference systems. The four methods come from the fact whether it is necessary to tune all the parameters (i.e. all the premise and the consequent parameters) of the critique and the actor or just tune their consequent parameters. The four methods are applied to three versions of the pursuit evasion games. In the first version just the pursuer is learning. In the second version, the evader uses its higher maneuverability and plays intelligently against a self-learning pursuer. In the final version, both the pursuer and the evader are learning. We evaluate which parameters are best to tune and which parameters have little impact on the performance.