Spherical VR cameras can capture high-quality immersive VR images with a 360° field of view. However, in practice, when the camera orientation is not straight, the acquired VR image appears tilted when displayed on a VR headset, which diminishes the quality of the VR experience. To overcome this problem, we present a deep learning-based approach that can automatically estimate the orientation of a VR image and return its upright version. In contrast to existing methods, our approach does not require the presence of lines or horizon in the image, and thus can be applied on a wide range of scenes. Extensive experiments and comparisons with state-of-the-art methods have successfully confirmed the validity of our approach.