In video coding standards such as Advanced Video Coding (AVC) and its successor High Efficiency Video Coding (HEVC), motion compensation is performed by partitioning each inter-predicted picture into square or rectangular regions. While HEVC introduces an efficient quad-tree based splitting of square-shaped coding blocks into prediction blocks by symmetric and asymmetric motion partitioning, the boundaries of natural moving objects can only be approximated by a fine block partitioning, resulting in a redundant representation of the motion and therefore a potential coding overhead. This paper studies the method of using a segmentation based partitioning of coding blocks into arbitrary shaped segments, where the segmentation is performed on coded reference pictures. Experimental results show that for cases where a reasonable segmentation can be obtained, bitrate reductions of around 2% can be achieved.