Machine learning is an important issue to researchers for several years. Reinforcement learning is a type of unsupervised learning which uses state-action combinations and rewards to interact with the environment. Q-learning a further, sub-division of reinforcement learning is now-a-days well-accepted algorithm for robots (machine) learning. However human beings learn in different ways. One of such learning is gradual learning which is mostly continuous in nature. This present paper uses gradual learning combined with Q-learning for light exploration. The first Q-table is randomly generated, but the next Q-tables are inter-dependent and gradually refined. Initial learning time may be high, but final learning time is lower and this proves the efficiency of this learning technique. Apart the convergence of the Q-learning is also established.