This paper proposes a new simple and efficient method to detect abrupt scene change based on only pixel values. Conventional pixel-based techniques can produce a significant number of false detections and missed detections when high motion and brightness variations are present in the video. To increase scene change detection accuracy yet maintain a low computational complexity, a two-phase strategy is developed. Frames are first tested against the mean absolute frame differences (MAFD) via a relaxed threshold, which rejects about 90% of the non-scene change frames. The rest 10% of the frames are then normalized via a histogram equalization process. A top-down approach is applied to refine the decision via four features: MAFD and three other features based on normalized pixel values - signed difference of mean absolute frame difference (SDMAFD*), absolute difference of frame variance (ADFV*), and mean absolute frame differences (MAFD*). Experimental results show that our method contributes to higher detection rate and lower missed detection rate while maintaining a low computational complexity, which is attractive for real-time video applications.