In recent years, it has become a focus to make use of the address recognition technology to improve the performance of mail sorting machines. The research in the postal address recognition, which extends the context relation from words to sentences with the use of the address information in post-processing, can effectively improve the recognition performance. In this paper, we propose a divide-and-rule method. The address is divided into high level address and low level address. For high level address, the similarity method with HLA database is presented. For low level address, the Syllable-based language model which applies to Pinyin is discussed. Then a new similarity method in multi-mode is introduced. After post-processing, for high level address, the hit rate rises from 87.63% to 95.12% while the accuracy rate declines by 2.67%, and for low level address, the hit rate increases from 58.16% to 91.81% while the accuracy rate decreases by 9.40%.