Past work has produced fairly accurate automatic pitch-accent detectors, but it has often been noted that the accent class of a word is highly dependent on word identity, with some words and word types usually being accented and others not. We argue that a good accent detector should not only have high overall accuracy, but also be able to distinguish between accented and unaccented variants of the same word. We report on experiments with several classifiers trained on a hand-labeled corpus, using a large set of acoustic features. Results show that while the classifiers have a high overall accuracy, they perform disappointingly on words with atypical accent status or whose prior accent status is more uncertain. We further report on attempts to improve the performance on these sub-tasks via feature selection and engineering of the training set.