Image captioning applications require prompt and precise caption generation, which can improve the accessibility and understanding capabilities of images. We utilize an actor-critic approach based on deep reinforcement learning and propose a two-fold approach to enhance the performance of the actor-critic approach. First, we propose a novel image-text matching module to compute the reward in image-matching, for the actor-critic model. This module enables more accurate and meaningful evaluations, contributing to improved caption generation. Second, we apply various training scenarios in reinforcement learning, strategically updating both the policy and value networks. The scenarios ensure more effective learning dynamics and lead to enhanced overall performance. To assess the efficiency of our approach, we employ the Microsoft COCO dataset. The experiments demonstrate the superiority of our method in terms of both speed and precision compared to the existing techniques.