Bag of Visual Words is one of the most effective image representations. One of the optimization methods for BoVW is the selection of the most informative visual words, which leads to more compact visual dictionaries and more accurate categorization. In this paper we investigate the problem of feature selection in the Bag of Visual Words framework. The main contribution is the presentation of two novel methods for visual word selection. The first one choses the features which are the best at separating one class from the rest (MFM1 one-vs-all). In the second method, the features which are the best at separating class pairs are selected (MSF6 one-vs-one). The effectiveness of the proposed methods is verified empirically on two different image datasets.