In this paper, we formulate and find distributed minimax strategies as an alternative to Nash equilibrium strategies for multi-agent systems communicating via graph topologies, i.e., communication restrictions are taken into account for the distributed design. We provide the conditions that guarantee the existence of the minimax solutions in the game. Finally, we present an off-policy Integral Reinforcement Learning (IRL) method to solve the minimax Riccati equations and determine the optimal and worst-case policies of the agents by measuring data along the system trajectories.