Novelty Search (NS) has been proposed as an alternative search approach for black-box optimization methods where the fitness function is replaced and only novel solutions are searched for. NS has been demonstrated as advantageous when the fitness landscape is highly deceptive and misdirects the search process towards local optima. In this research we test the efficacy of NS in comparison to a purely objective based approach and a hybrid approach that combines NS and a fitness function in combination with two behavior characterization schemes. The task is non-Markovian double-pole balancing. Results indicate that the success of NS strongly depends upon the behavior characterization scheme used, given that NS performed the best under one scheme and relatively poorly under the other scheme.