In this paper, we discuss the role of robustness to geometric conformity versus specificity in machine learning. The key observation made here is that object variation due to appearance and due to geometric deformation are often, for good reasons, intermixed in typical object detection applications. In the paper, we consider a whole range of differently specific object detectors. It is shown that such detectors vary in their robustness to geometric deformation and also their specificity. Such detectors can then be used in a cascade, where coarse detectors operate on a less-specific and more robust scale. This makes it possible to use coarse sampling of the space of geometric transformations. Further on more-specific and less robust detectors are used. This requires as input the detections at a coarser scale combined with an optimization search step. In the paper, it is also discussed how such detectors can automatically be obtained from a coarsely defined database of ground truth.