Rheumatologists determine treatment plan based on the inflammation of knee joints affected by arthritis. Extraction of the inflamed region or hotspot from the knee thermogram is the prerequisite for grading of inflammation and classification of different arthritis. In this paper, we propose an automatic method for extracting the inflamed region from the knee thermograms. We propose an ensemble technique to arrive at a consensus segmentation of the hotspot region. We have used variation of information based information theoretic approach to generate consensus segmentation. The fusion of multiple segmentation maps is achieved using local search based greedy iterated conditional modes algorithm to obtain final segmentation result. Experiments show that our proposal scores significantly better in detecting hotspots in more than 50 inflammatory knee thermograms.