Accurate localization systems are still a bottleneck for Unmanned Underwater Vehicles (UUVs). In recent years, fiducial markers have become a readily available, low-cost option. However, an in-depth analysis of marker detection accuracy in the underwater domain has yet to be performed. We propose a methodology to evaluate fiducial marker systems, namely the popular AprilTag system, in experiments. Our study especially focuses on aspects crucial for highly agile micro underwater robots, such as dynamic motions and the calibration medium. This class of robots is typically extensively studied in research tanks which motivates a first focus on clear-water settings. However, the proposed method and the findings can be transferred to similar scenarios. We demonstrate the importance of calibrating underwater and that the detection accuracy decreases linearly with camera distance and could therefore easily be compensated for. Moreover, we identify a suitable camera that maximizes the detection rate during highly dynamic motions. In sum, this work is an initial step towards application-relevant design strategies for designing low-cost, accessible localization systems for agile, mobile robots.