Semantic aware sport image resizing jointly using seam carving and warping

Abstract

In content aware image resizing, saliency map or gradient is usually used to determine the important regions of images. But for sport images such as basketball and football images, these methods may falsely classify parts of court fields as unimportant regions, while parts of grandstands as important regions. Such results are not consistent with human perception. In this paper, a semantic aware image resizing approach is proposed. We extract the semantic information automatically. We segment the court fields as important regions and detect the boundary of court fields as the semantic edges. Considering the complementary characteristic of discrete image resizing approaches such as seam carving and continuous approaches such as warping, seam carving and warping are jointly used in our scheme. We define the Semantic Weight Function (SWF) based on the semantically important regions. Then semantic aware seam carving (SASC) is proposed based on the SWF. Next we define the Deformation of Semantic Edges (DSE) to assess the image deformation caused by seam carving. Finally seam carving and warping are joined using the DSE. We compare our approach with approaches like scaling, seam carving and semantic aware seam carving (SASC). Experimental results show that our approach preserves more semantically important regions with less deformation. Our approach also preserves the aspect ratio of key objects.