The human vision system is highly adapted to extract structural information from the viewed scenes. The irregularity of point clouds makes the extraction of structural information containing both color and geometry an important challenge for point cloud quality assessment (PCQA). This paper proposes a point structural information (PSI) network (ψ-Net) for no-reference PCQA. Firstly, a PSI module is proposed to map the position vectors of neighboring points to weights for the calculation of color and geometric structure information. Secondly, a dual-stream network is presented to introduce distortion-related features for PCQA. Experimental results show the effectiveness of the proposed method.