In this paper, we propose a novel decomposition method to segment multiple object regions simultaneously in cluttered videos. This method formulates object regions segmentation as a labeling problem in which we assign object IDs to the superpixels in a sequence of video frames so that the unary color matching cost is low, the assignment induces compact segments, and the superpixel labeling is consistent through time. Multi-object segmentation in a video is a combinatorial problem. We propose a binary linear formulation. Since the integer linear programming is hard to solve directly, we relax it and further decompose the relaxation into a sequence of much simpler max-flow problems. The proposed method is guaranteed to converge in a finite number of steps to the global optimum of the relaxation. It also has a high chance to obtain all integer solution and therefore achieves the global optimum. The rounding of the relaxation result gives an N-approximation solution, where N is the number of objects. Comparing to directly solving the integer program, the novel decomposition method speeds up the computation by orders of magnitude. Our experiments show that the proposed method is robust against object pose variation, occlusion and is more accurate than the competing methods while at the same time maintains the efficiency.