As the complexity of the specifications that must be met by a system increases, hierarchical control protocols that merge control and planning decisions at multiple levels of abstraction become necessary. For such hierarchical reasoning, a suitable finite-state abstraction for dynamical systems evolving over continuous state spaces may be needed. The implementation of existing controllers derived using a finite-state abstraction often require that the current continuous state be known exactly, in order to guarantee that the required transitions in the finite-state abstraction occur. When the measurements are partial or noisy, the true state is unknown, and these controllers cannot be implemented. We propose an abstraction that can be used to overcome the uncertainty in the state resulting from imperfect measurement, at the cost of providing only probabilistic guarantees. The abstraction is based on the filter used to maintain an estimate of the true state. We show how the abstraction can be used to create a time-varying policy which maximizes the minimum probability that a target discrete state is reached in finite time from any initial state.