In the natural and practical scenario, the captured video sequence under bad weather situations or low light conditions often suffers from poor visibility and low-contrast problems. This hurts the performance of the high-level processing, e.g. object tracking or recognition. In this paper, we develop an FPGA-based low-visibility enhancement accelerator for video sequence by adaptive histogram equalization with dynamic clip-threshold (AHEwDC) which is determined by the visibility assessment. The main goal is to improve the low visibility with high image quality for both hazy and low-light video sequences in real-time. Firstly, a concept to quantify the visual perception based on supervised learning is to estimate the visibility score. Then, to avoid the problem of noise amplification in the conventional method, we propose a visibility assessment model to find an optimal clip-threshold. The contrast energy of gray channel, yellow-blue channel and red-green channel, average saturation, and gradients are statistical features in the model to describe the visibility of an image. Finally, to meet the speed requirement for video sequence processing, a specified hardware architecture for both visibility assessment and AHEwDC is implemented on FPGA. Besides, a mean spatial filter for cumulative distribution functions (CDFs) of the AHE is developed for suppressing the noise caused by a single-color local region. The demonstration system on the DE1-SoC platform with the Intel Cyclone V FPGA device with the max working frequency of 75.84 MHz is capable of processing 30 fps FHD ( $1920\times 1080$ ) video.