Collaborative monitoring of large infrastructures, such as military, transportation and maritime systems are decisive issues in many surveillance, protection, and security applications. In many of these applications, dynamic multi-agent systems using reinforcement learning for agents' autonomous path planning, where agents could be moving randomly to reach their respective goals and avoiding topographical obstacles intelligently, becomes a challenging problem. This is specially so in a dynamic agent environment. In our prior work we presented an intelligent multi-agent hybrid reactive and reinforcement learning technique for collaborative autonomous agent path planning for monitoring Critical Key Infrastructures and Resources (CKIR) in a geographically and a computationally distributed systems. Here agent monitoring of large environments is reduced to monitoring of relatively smaller track-able geographically distributed agent environment regions. In this paper we tackle this problem in the challenging case of complex and cluttered environments, where agents' initial random-walk paths become challenging and relatively nonconverging. Here we propose a multi-agent distributed hybrid reactive re-enforcement learning technique based on selected agent intermediary sub-goals using a learning reward scheme in a distributed-computing memory setting. Various case study scenarios are presented for convergence study to the shortest minimum-amount-of-time exploratory steps for faster and efficient agent learning. In this work the distributed dynamic agent communication is done via a Message Passing Interface (MPI).