Noisy network is a typical method for the exploration of reinforcement learning by adding noises in parameter domain. However, the slow reduction of noises will lead to an unstable policy. In this paper, fuzziness is introduced to the noise network to achieve the stable policy without extra hyper-parameters added. Experiments on gym Atari games show that the proposed fuzzy noisy DQN (FNDQN) algorithm can achieve stable policy, as well as the higher scores for the agent.