This paper proposes a new architecture of video combiner for multipoint video conferencing. The proposed video combiner first identifies the active and talking conference participants from the multiple incoming video bitstreams by computing their corresponding motion activities and level of audio signals. By reducing the frame rates of inactive sub-sequences, more bits are reallocated to the active sub-sequences such that higher frame rates are obtained to produce a better visual perception as well as a smoother motion. We also propose a low-complexity and high-performance frame-skipping transcoder which is achieved by a direct summation of the DCT coefficients for non-moving macroblocks and an adaptive motion vector refinement for motion compensated macroblocks. The new transcoder is then used to realize the multipoint video conferencing and some results are presented to show the improvement in performance due to our proposed architecture.