With the growing popularity of 3D content and virtual reality applications, effective no-reference stereoscopic image quality assessment (NR-SIQA) methods have become increasingly important. In this paper, we propose a convolutional neural network (CNN) SIQA model based on binocular rivalry and fusion mechanisms of the human visual system (HVS). In order to get a better representation of binocular information, we propose Two-Stage Enhanced Fusion Module (TSEFM) that consists of two stages for monocular features enhancement and binocular features fusion, respectively. Given the dynamical characteristics of binocular rivalry phenomenon, the proposed Content-Aware Binocular Rivalry Fusion Module (CABRFM) dynamically and adaptively adjusts its output based on the input content. Additionally, considering that feedback mechanism of HVS is indispensable and significant, we introduce feedback connections during feature aggregation to realize the guidance of high-level features to low-level features. Extensive experimental results demonstrate the superiority of our method over state-of-the-art metrics, showcasing its excellent performance.