Detecting moving objects in dynamic environments is precisely essential in autonomous driving. Existing object detection methods using point clouds have difficulties in distinguishing moving and static objects in dynamic environments. Motivated by the optical flow method widely used in image-based dynamic object perception, we propose semantic scene flow-assisted moving object segmentation (SSF-MOS), a unified framework that incorporates semantic information and ego-motion estimation in MOS. SSF-MOS first detects and excludes absolutely static objects, such as poles and roads, by applying the semantic segmentation method. Subsequently, the proposed SSF estimation method computes the motion vectors between consecutive point clouds and predicts the motion state (moving or static) of each point; furthermore, SSF-MOS calibrates the results of moving points by considering the ego-motion of autonomous vehicles. We directly introduce semantic information in the decoupled framework for more accurate results and convenience of upgrades. The extensive experiments show that the proposed SSF-MOS achieves a competitive performance of 0.701 mIOU compared with other state-of-the-art methods on the public dataset SemanticKITTI.