With the development of 3D technology, there is an urgent demand for accurate and efficient perceptual quality evaluation of stereoscopic images. In this paper, we propose a primary-visual-processing-mechanism-based metric (PVPM) for stereoscopic image quality assessment (SIQA). Considering binocular and monocular cells in primary visual cortex, we classify image regions into binocular and monocular ones. Inter-view difference of dominance appears in perception of binocular cells under asymmetric quality, so local quality of left and right binocular regions are weighted by stimuli strength indexes to obtain joint binocular local quality. The overall quality is generated through pooling of binocular and monocular local quality, in which visual sensitivity and region sizes are taken into account. Experimental results demonstrate that PVPM can achieve higher accuracy as well as faster speed compared to state-of-the-art SIQA metrics.