Background subtraction is an example of a moving object detection technique that uses machine vision systems. Conventional moving object detection methods need complicated thresholds for background modeling to address changes in illumination. This paper proposes a novel background modeling approach without thresholding based on a bit-planes method, which fully utilizes color characteristics through spatial and temporal-based improvement. The proposed idea is effective and efficiently solving for shadow disturbance and brightness changes. We evaluate our proposed method using several challenging indoor and outdoor sequences from the CDNET 2014 dataset. The experiments show that the proposed idea typically achieves a higher rate of detection accuracy than those of the current state-of-the-art approaches.