Parsing video programs into program segments is useful in retrieval of individual segments and video summarization. Many video classes show structure in them that can be effectively model using finite-state automata (FSA). Each of the video segments, such as newscaster sequence, weather sequence etc., becomes a node in FSA. The transition is fired from one node to another node based on arc conditions, which can easily be obtained by employing statistical methods on classified data. Modeling with FSA avoids the use of complex rule-based systems. Experimental results presented by the FSA approach for more than 8 hours of video data show an accuracy of 88% on recognition of components of news video.