Due to the highly uncertainty of pedestrian movement on the road, understanding pedestrian behavior remains a complex and challenging task in the field of autonomous driving. While numerous studies have been conducted on pedestrian action recognition for intention prediction, we focus on pedestrian gesture detection, an aspect that has received little attention. Pedestrians often use certain arm gestures to express their intentions to oncoming vehicles. Detection of pedestrian gestures assists autonomous vehicles in understanding pedestrian intentions like human drivers. We propose a skeleton-based approach for pedestrian arm gesture detection and recognition. A pose estimation algorithm is applied to extract skeleton points of pedestrians. The angle and relative position between the pedestrian's arm and body are extracted for online arm gesture detection. Then the angle and moving pose descriptor is adopted for gesture recognition with support vector machines as the classifier. Experimental results on multiple datasets show that our proposed method outperforms other similar work.