The escalating automation of operations in manufacturing systems has seen a notable rise in the utilization of automated guided vehicles (AGVs) within automated material handling systems. AGVs, reliant on battery power, necessitate strategic charging policies to avert battery depletion during operations. However, prevailing heuristic approaches often yield inefficiencies. This study formulates the AGV charging problem as a Markov decision process (MDP) model, considering the uncertainty and geometric information of the environment. Relative value iteration has been implemented to optimize the MDP model. The proposed charging policy undergoes rigorous analysis and comparison with existing heuristics through simulation experiments. This process establishes its efficacy in advancing AGV charging efficiency.