A Hough transform-based method for action detection can achieve robustness to occlusions because the method casts votes for action classes and spatio-temporal action positions based on the visible local features of partially occluded actions. However, each local feature is prone to a false vote. This paper focuses on the trend of past votes to curb the influence of false votes by extending conventional Hough forests to sensing that trend. Our proposed method, called trendsensitive Hough forests, learns a voting trend model that can be used to discriminate between correct and false votes and calculate the confidence of them. We experimentally confirmed that it outperformed action detection accuracy of conventional Hough forests.