Non-asymptotic and asymptotic analyses on Markov chains in several problems | IEEE Conference Publication | IEEE Xplore