Limitations on hardware resource consumption and throughput make the use of non-binary low-density parity-check (NB-LDPC) codes challenging in practical applications. This brief first proposes a novel check-node (CN) decoding algorithm called Hamming-distance trellis min-max (H-TMM) to reduce the complexity introduced by searching for two minimum values in previous TMM-based algorithms. Then, by taking advantage of the proposed algorithm and the number appearances of reliable values, a high-performance H-TMM-based NB-LDPC decoder architecture is presented. Experiments on the 32-ary (837, 726) NB-LDPC code confirmed that the proposed decoder can obtain a high throughput on less hardware resources compared with the state-of-the-art works while maintaining a competitive error-correcting performance. In particular, the proposed CN unit architecture reduced hardware resources by almost half compared to that in the latest decoder.