Recently, convolutional neural network (CNN) has been successfully utilized in the terrain classification of polarimetric synthetic aperture radar (PolSAR) images. However, most CNN-based models are currently limited to handle 2-D real-valued inputs, and therefore, the physical scattering mechanism contained in the complex-valued (CV) covariance/coherency matrix cannot be extracted effectively. For this reason, CV 3-D CNN (CV-3D-CNN) is proposed for PolSAR image classification. Compared with CNN, CV-3D-CNN simultaneously extracts hierarchical features in both the spatial and the scattering dimensions by performing 3-D CV convolutions, thereby capturing the physical property from polarimetric adjacent resolution cells. Experiments on real PolSAR images classification demonstrate the effectiveness and the superiorities of CV-3D-CNN and illustrate that CV-3D-CNN can deal with scattering characteristic in a more complete manner and achieve better performance in PolSAR image classification.