Many papers are published every year containing new methodologies for brain tissue segmentation in magnetic resonance images. The evaluation of these methods is fundamental to understand their behavior and to observe their weak and strong points. Even though improvements have been proposed, the analysis of the segmentation results can still lead to incorrect conclusions. This paper contains an investigation of the state-of-the-art in brain tissue segmentation evaluation, which includes tissue classification or segmentation, handling partial volume effect, and evaluation metrics. It uncovers previously unnoticed pitfalls and proposes standard procedures to avoid them. Experiments show that the proposed evaluation strategy gives a better insight about the method's strong and weak points.