With the large and increasing amount of visual information available in digital libraries and the Web, efficient and robust systems for image retrieval are urgently needed. A compact color descriptor scheme and an efficient metric to compare and retrieve images is presented. An image adaptive color clustering method, called fuzzy color signature, is proposed. The original image colors are mapped into a small number of representative colors using a peaks detection function derived from the color distribution. Fuzzy color signatures are then used as image descriptors. To compare image descriptors the earth mover's distance is used. Several experiments have been conducted to assess the performance of the proposed technique.