In order to increase coding efficiency, a perception-aware H.264/AVC encoder is proposed in this paper. With a different perception models for intra-frames and inter-frames, the initial quantization parameter (QP) is perceptually adjusted to remove perceptual redundancy. Moreover, the associated hardware architectures of the whole encoder and the perception analysis engine are also proposed with hardware sharing and pipelining design techniques. Experimental results show that our encoder achieves about 6-27% bit-rate saving in the QP range of 28-36. Furthermore, the hardware cost of the perception analysis engine is 120K gates, which is acceptable to be integrated into an H.264/AVC encoder chip.