Due to the lack of the annotated nighttime images, general image segmentation models trained on the daytime image dataset do not perform well in nighttime scenes. The difference of the illumination condition and the difficulty to obtain the position information between daytime and nighttime makes the nighttime image segmentation tough. As a consequence, this paper proposes an end-to-end nighttime segmentation network based on the following two points: 1) Utilizing illumination adaptation with the different illumination condition on the daytime or nighttime to close the distribution gap at the feature map level; 2) With the prior information about the position of each object in the outdoor scene, some classification errors could be corrected by incorporating the self-attention mechanism. The scheme is tested on the open-source nighttime dataset Dark Zurich and night driving, with a 2.5% improvement compared to the base segmentation network.