Recently, many researchers have studied fiducial-based tracking of deformable nonrigid surfaces for projection mapping purposes, but efficient handling and representation of fine deformations and sharp contours in such surfaces still remain a challenge. On top of existing fiducial-based projection mapping methods, much effort is still required to achieve a projection mapping that is free from geometric errors, misalignments, and artifacts in the areas that contain such deformations. According to our observation, this is due to the fact that methods relying on fiducial markers have used markers that sparsely represent the projection surface, are large in size, and projector–camera registration is not accurate enough. We propose a method that enables us to accurately track a deformable nonrigid surface and project textures onto it, ensuring that no visible misalignments occur. We achieve this by proposing a fiducial marker design followed by an adaptive method for fiducial selection to adequately model the deformations in a nonrigid projection surface. Our proposed marker is scalable in size and provides several advantages, such as robust detection under strong deformations and support for adaptively varying fiducial marker density, which enables us to handle the aforementioned deformations with increased accuracy. We have demonstrated our results using a sheet of paper and a cloth under several geometric deformations. Our experimental results show compelling results in terms of geometric accuracy.