The conventional studies on different types of remote sensing (RS) images classifications are conducted separately. Thanks to the powerful potential of deep learning to automatically learn features from data, exploring a unified method is possible. Moreover, recent research shows that sparse and low-rank representations can convey valuable information for patterns classification. Therefore, this paper presents a two-stream heterogeneous RS images unified interpretation network (HRSIUI-Net). One stream is to transfer the pre-trained fully convolutional network to learn deep multi-scale spatial features of RS data. The other stream is to employ a subspace learning based on graph embedding to learn the sparse and low-rank subspace representations of high-dimensional features. And then, two streams of learned subspace features are integrated for classification combined with an SVM. The experimental results on two typical RS data indicate that HRSIUI-Net can achieve competitive performance.