Effect of Maximizing Recall and Agglomeration of Feedback on Accuracy

Abstract

This paper studies multiple aspects of modeling user preference in a heterogeneous environment, where different individuals describe their level of comfort with the temperatures in different rooms in a building. The study shows that sampling based on fuzzy clustering provides the best approach for addressing the imbalance resulting from limited feedback points. Another issue addressed in the paper is a comparison of models based on agglomerated dataset for the entire building versus datasets for individual rooms. Ideally, personalized models for individual rooms should provide the best models. However, the number of feedback points for individual rooms is much smaller resulting in even larger imbalance in data. In many diagnostic situations in engineering and health sciences, recalling the critical decisions is more important than the prediction accuracy. The paper studies the quality of modeling by maximizing recall versus maximizing the AUC of models.