We propose an ultra-wideband-based (UWB) localization system that achieves high accuracy through nonparametric estimation of measurement probability densities and explicit modeling of antenna delays. This problem is difficult because non-line-of-sight conditions give rise to multimodal errors, which make linear estimation methods ineffective. The primary contribution in this paper is an approach for both characterizing these errors in situ and an optimization framework that recovers both positions and antenna delays. We evaluate our system with a network of 8 nodes based on the DecaWave DWM1000 and achieve accuracies from 3 cm RMSE in line-of-sight conditions to 30 cm RMSE in non-line-of-sight conditions. Collecting measurements and localizing the network in this manner requires less than a minute, after which the realized network may be used for dynamic real-time tracking.