A novel siamese autoencoder visual odometry system named SAEVO is proposed in this paper. SAEVO can jointly estimate the 6-DoF pose and the depth using deep neural networks trained with monocular clips only. The main idea of the proposed method is an unsupervised deep learning scheme that combines siamese networks with auto-encoder for multi-scale matching to estimate ego-motion. Also, two unsupervised losses are designed to align extracted features from the siamese autoencoder networks. A system overview is shown in Fig. 1. The experiments on KITTI and CityScapes datasets demonstrate the SAEVO achieves good performance in terms of pose and depth accuracy, and competitive performance to state-of-the-art methods.