Agents in open multi-agent systems (MAS) need means for locating other agents with which they may collaborate. To address this need, several agent location mechanisms were suggested. Two major approaches dominate agent location mechanisms: a centralized approach using middle agents, and a distributed, peer-to-peer approach. Agent designers, when designing agents to be part of open MAS, should consider these approaches, to provide the agents with appropriate agent location capabilities. However, selecting an agent location approach, let alone a specific solution, is a nontrivial task. In this study we address this difficulty. We perform a systematic comparative evaluation of agent location approaches. We measure the performance of these approaches subject to various MAS configurations. We draw conclusions regarding the conditions in which each approach is preferable. Prior evaluations fall short in addressing realistic MAS settings. In particular, our evaluation is the first to examine scalability of agent location mechanisms in terms of both system size (thousands of agents) and network distribution (over multiple hosts). We present advantages and shortcomings of the examined approaches.