Over the years, Automated Image Analysis Systems (AIAS) have been developed for pavement surface analysis and management. Pavement distress segmentation is a key issue throughout the entire process of analyses. In this paper, an adaptive approach for pavement distress segmentation based on Genetic Algorithms is proposed. After the pavement images are captured, an objective function is defined and maximized by applying information theory to choose the optimal threshold for segmentation. Regions corresponding to distresses are represented by a matrix of square tiles. The vertical and horizontal distress measures along with the total number of distress tiles are then calculated providing input into a three-layer feed-forward neural network for a type classification. The proposed analysis algorithm is capable of enhancing the pavement image, extracting the distress from the background, and analyzing its type. To validate the system, actual pavement pictures were taken from both highway and local road pavements. The experimental results demonstrate that the proposed model works well for pavement distress detection and classification.