In this work, reinforcement learning is used to develop a position controller for an underactuated nature-inspired Unmanned Aerial Vehicle (UAV). This particular configuration of UAVs achieves lift by spinning its entire body contrary to standard multi-rotors or fixed-wing aircraft. Deep Deterministic Policy Gradients (DDPG) with Ape-X Distributed Prioritized Experience Replay was used to train neural network function approximators that were implemented as the final control policy. The reinforcement learning agent was trained in simulations and directly ported over to real-life hardware. Position control tests were performed on the learned control policy and compared to a baseline PID controller. The learned controller was found to exhibit better control over the inherent oscillations that arise from the non-linear dynamics of the platform.