LiDAR point cloud is an important component of 3-D reconstruction, and high-precision 3-D point clouds provide accurate depth and scene structure. However, dynamic objects can form ghosting in the point cloud during data acquisition, causing data corruption. Most researchers have used the occlusion relationship between dynamic and static objects to achieve simple trajectory removal of dynamic objects in outdoor open scenes. However, in indoor scenes, due to the complex occlusion relationship, it is not possible to ensure the fine-grained, efficiency, and accuracy of the removal to achieve practical application levels. Here, we proposed an offline point cloud dynamic object removal method based on time visibility. We believe that dynamic objects can be regarded as a more obvious noise point, and it is feasible to distinguish dynamic and static objects from the time dimension under the premise of their unstable observation. Therefore, we proposed a multistage method for dynamic object removal. Specifically, we use hash-encoded voxels to express local space, encode local space’s time sequence occupancy count as a unified vector, and summarize comprehensive indicators of observation continuity and repeatability. This metric is used to evaluate the state of voxels, and then remove the dynamic part. In addition, we proposed a density-based binary classification method for more fine-grained removal tasks. Finally, we have validated the proposed algorithm’s robustness, advancement, and efficiency in multiple challenging indoor scenes.