Model-Based Interpolation for Continuous Human Silhouette Images by Height-Constraint Assumption

Boundary information is crucial for defining the shape of an object or a person as a first approximation, and is used extensively for quantitative analysis in various fields such as fluids, cell biology, medical images, and human video analysis. In particular, the analysis of human silhouettes is vital in numerous applications. However, when videos are captured at frame rates that are lower than the speeds of the target human movements, the analysis accuracy of the corresponding videos decreases owing to the lack of in-between boundary information of the pair of successive boundaries. A reliable boundary-tracking technique for interpolating in-between human silhouettes is necessary to compensate for such an information loss; however, a suitable method for human video analysis has not yet been developed. To this end, we propose a novel model-based boundary-tracking method based on the height-constraint assumption, in which the positions of the body points are assumed to change insignificantly along the height (or vertical) direction during walking. In the case of videos of humans walking and jogging, the proposed method can perform frame interpolation with substantially higher accuracy than that of existing methods: e.g. the normalized square error of the proposed method is about 1.4~1.9 fold smaller than those of existing methods in the walking case of 7.5 fps input frame rate. Moreover, it is confirmed that, although the height-constraint assumption is less effective at diagonal-view angles than at lateral-view angles, the proposed method with the additional projective transform can perform more accurate interpolation than that with only the height-constraint assumption, even in the case of diagonal-view angles. Furthermore, it is preliminarily revealed that preprocessing of the proposed method leads to improved accuracy of forensic gait-based human identification and visual hull-based 3D reconstruction under low frame rate conditions. It is expected that the proposed method will also be effective for application in other fields, such as action recognition, kinesiology, and sports video analysis.