The problem of incorporating spatial information to the bag-of-visual-words model for image classification is addressed in this letter. To incorporate such information, we propose to encode the global geometric relationships of the visual words in the 2D image plane in a scale- and rotation-invariant manner. This is established by measuring scale- and rotation-invariant geometrical properties given by triangles of identical visual words. Experimental results demonstrate that our proposed method is more robust to changes in scale and image rotations than the bag-of-visual words model on a butterfly and fish dataset.