Special-purpose hardware to solve optimization problems formulated as Ising models has generated great excitement recently. Despite a large diversity in hardware, most solvers employ standard variations of the classical (simulated) annealing (CA) algorithm. Here, we show how powerful replica-based Parallel Tempering (PT) algorithms can significantly outperform CA, using FPGA-based probabilistic computers. Using a massively parallel (graph-colored) architecture, we implement the Adaptive PT (APT) algorithm, generating problem-dependent temperature profiles to equalize replica swap probabilities. We benchmark our p-computer against analytical results from classical Ising theory and use our machine to solve spin-glass instances formulated as hard optimization problems. APT outperforms heuristic choices of temperature profiles used in conventional PT and a replica-based version of CA. Our machine provides 6,000X speedup over optimized CPU, with orders of magnitude further speedup projected for scaled implementations. The developed co-design techniques may be useful for a broad range of Ising machines beyond p-computers.