3D point cloud is often affected by sensor noise, environmental interference, and incomplete collection, resulting in noise, missing, and anomalies in the point cloud. Existing work in handling such data primarily focuses on the coordinate information of the point cloud, overlooking its local structure and the interrelation between points, thereby reducing the accuracy of model predictions. To enhance the ability to capture point cloud features, we propose a high-precision and robust 3D point cloud analysis model called PointMLFF. Specifically, we introduce a 3D surface-based point local feature extractor, capturing the local geometric features of the point cloud by approximating the Taylor series to construct local spatial geometries. This method preserves both the absolute position information and the local shape information of the point cloud. Additionally, we propose a multi-level key-point attention module based on deep features, which constructs a point embedding space capable of perceiving abnormal changes in the point cloud by calculating the key-neighbor point attention and inter-key point attention in the feature space, significantly improving the model’s robustness. Extensive experiments show that PointMLFF outperforms most advanced methods in various downstream tasks. Notably, our method achieves a high classification accuracy of 88.9% on the challenging ScanObjectNN and surpasses others on abnormal point clouds. The visualization of partial segmentation results closely resembles the actual scenarios.