In the last few years, the field of mobile robotics has made lots of advancements. These advancements are due to the extensive application of mobile robots for autonomous exploration. Mobile robots are being popularly used for applications in space, underwater explorations, underground coal mines monitoring, inspection in chemical/toxic/ nuclear factories etc. But if these environments are unknown/unpredictable, conventional/ classical robotics may not serve the purpose. In such cases robot learning is the best option. Learning from the past experiences, is one such way for real time application of robots for completely unknown environments. Reinforcement learning is one of the best learning methods for robots using a constant system-environment interaction. Both single and multi-agent concepts are available for implementation of learning. The current research work describes a multi-agent based reinforcement learning using the concept of behaviour-based robotics for autonomous exploration of mobile robots. The concept has also been tested both in indoor and outdoor environments using real-time robots.