We report an experiment in which participants compared the size of a visual sphere to a haptic sphere in VR. The sizes from two modalities were either congruent or conflicting (with different disparities). Importantly, three standard references (small, medium and large) for haptic sizes were used with the method of constant stimuli. Results show a dominant functional priority of the visual size perception. Moreover, observers demonstrated a central tendency effect: over-estimation for smaller haptic sizes but under-estimation for larger haptic sizes. The results are discussed in the framework of adaptation level theory for haptic size reference, and provide important implications for the design of 3D visuo-haptic human-computer interaction.