In this paper, we propose a practical method for removing specular artifacts on the epicardial surface of the heart in robotic laparoscopic surgery while preserving the underlying image structure. We use freeform temporal registration of the non-rigid surface motion to recover chromatic information saturated by highlights. The diffuse and specular image components are then separated by shifting pixel intensities with respect to chromaticity gathered from the spatio-temporal volume. Results on in vivo data and reconstructions of 3D structure from the diffuse images show the potential value of the technique.