Twin support vector machines are a powerful learning method for binary classification. Compared to standard support vector machines, they learn two hyperplanes rather than one as in standard support vector machines, and work faster and sometimes perform better than support vector machines. However, relatively little is known about their theoretical performance. As recent tightest bounds for practical applications, PAC-Bayes bounds are based on a prior and posterior over the distribution of classifiers. In this paper, we study twin support vector machines from a theoretical perspective and use the PAC-Bayes bound to measure the generalization error bound of twin support vector machines. Experimental results on real-world datasets show better predictive capabilities of the PAC-Bayes bound for twin support vector machines compared to the PAC-Bayes bound for support vector machines.