Effective detection of crowd escape behavior in public places is a challenge in computer vision. Although some studies on such detection have been done in recent years, no complete solution satisfies the current application demand. This work develops an improved neural network to detect crowd escape behavior presented in video monitoring systems. In the neural network, luminance change caused by crowd activity is first collected; second, the gathered excitation from each image pixel is mixed with the delayed excitation from the related neighboring counterpart in a specific proportion, relying upon the inspiration of visual information integration in the mammalian’s retina; finally, an unique adaptive threshold scheme is designed to regulate the discharge excitation of the neural network in order to perceive the process of the crowd escape behavior. Experiments have validated that the neural network is effective for detecting the escape behavior of crowd, based on a public video data set of crowd escape events.