The purpose of nonuniformity and bad pixel correction is to provide a more reliable foundation for subsequent image processing and target detection. Existing correction methods generally struggle to balance the contradiction between oversmoothing and residual noise. Particularly, oversmoothing can easily filter out texture details and dim small targets. Based on the multiframe response model of the infrared focal plane array (IRFPA) detector, we propose a two-stage 3-D residual fully convolutional network (FCN) for correction factor estimation, integrated with an oversmoothing suppression mechanism. The proposed method designs two 3-D subnetworks to estimate the gain correction factors and offset correction factors, respectively. For the correction factor pre-estimation tensors outputted by the two subnetworks, an interframe averaging after outlier removal is applied to suppress oversmoothing. Ultimately, using multiplication and addition structures, the final estimated values of the gain and offset correction factors can be utilized to obtain the corrected images. Experimental results indicate that the proposed method exhibits substantial generalization capabilities toward different intensities of nonuniformity pixelwise fixed mode noise and can effectively correct the bad pixels of real infrared images while suppressing oversmoothing and maintaining the image details such as dim small targets well. Overall, as a method that combines the model-driven and the data-driven, our method possesses strong theoretical interpretability and superior performance.