We used the same reproduction method of stars and experimental stimuli as in the previous study [17]. Therefore, in this subsection, we briefly explain the reproduction method and star fields used as experimental stimuli.

We performed a psychometric experiment to evaluate the perception of faithfulness and preference of star-image reproduction in a planetarium for 9 types of projection patterns, as summarized in Table I. In the faithfulness evaluation, the observers evaluated the faithfulness using an opposite word pair (“faithful” / “non-faithful”) and five integer levels from −2 to +2 compared with the actual starry sky in their memory, and wrote their evaluation values down on answer cards based on a 5-point Likert scale. The meanings of the evaluation levels were: −2 (not faithful), −1 (slightly not faithful), 0 (neutral), +1 (slightly faithful), and +2 (faithful). In the preference evaluation, the observers evaluated using another opposite word pair (“preference” / “non-preference”) with the same five integer levels from −2 to +2 without a comparison target. The answer task was performed in complete darkness, only with the projected star image to maintain the dark adaptation; there was no other bias to discriminate against particular answers. In the evaluation, there was no designated fixation point; the observers were able to freely observe the star image. Therefore, they could well judge the color and brightness of the whole projection stimuli using the foveal vision by the cones and peripheral vision by the rods. A snapshot image of the illuminated dome captured using a fish-eye lens is shown in Figure 3.

This psychometric experiment was performed using the dome of the planetarium. Each star pattern was projected to the position of the oval mark in the figure. The diameter of the dome screen was 23 m; the zenith of the dome screen was slanted 15° to the front. There was no other illumination in the space where the experiments were performed, aside from that of the projected starry sky image. We confirmed that the room appeared completely dark. It was not possible to verify the low light level using the CS-2000 as it was too dark to measure (< 0.003 cd/m2). The average viewing distance between the observers and center of the projected star-field image was approximately 10.7 m (viewing angle: 37.3∘); it slightly varied according to the seat position.

A total of 47 observers participated in this experiment. Biased observers participated in our previous study in terms of gender and age; in contrast, in this experiment, various types of observers participated. As shown in Figure 4, 33 male and 14 female observers with different ages participated. A requirement for the observers was that they had experience in astronomical observation. Two experiments were conducted, for a set of 26 observers, and a set of 21 observers, respectively, in order to maintain a similar viewing distance between the observers and the projection on the planetarium dome. The experimental process was the same for both sets of observers. Once the observers took a seat in the planetarium, the illumination of the dome was turned off. We confirmed that the brightness of the dark dome was lower than a magnitude of 23.0 using the sky quality meter. Most stars with brightness higher than a magnitude of 7.4 reproduced by the projection could be perceived as the magnitude limit for observation in the 23.0-magnitude darkness was approximately 7.0 [18].

Prior to the psychometric experiment, the observer received instructions for the evaluation method and exercised using a practice pattern for 20 min. It was assumed that the observers had completed the dark adaptation by this time. The five-point evaluation system for rating the faithfulness (or preference) of the projection on a scale of − 2 to +2 was explained to the observers. The observers were provided with an answer card to record their ratings. During the experiments, the observers were not allowed to talk to each other or to move from their appointed seat. The experiment was proceeded by oral instructions using a microphone in the dark dome. In this experiment, 10 randomly projected patterns (including standard patterns twice to confirm reproducibility) were used. Note that the first Std pattern was projected first. The observers evaluated the faithfulness and preference of each pattern (each of them within 15 s) after observing the star image for 30 s. For the observation, we set a time period of 30 s as a previous study reported that the detection limit of the brightness in the dark environment became constant after 15 s during the experiment [21]. A short break of few minutes was set between the pattern evaluations to reset the last evaluation on the observers’ perception and exchange the projection pattern. The observers were given a ten-minute break after half of the experimental patterns had been evaluated, and the experiment was resumed after a period of dark adaptation. All experiments were performed in accordance with The Code of Ethics of the World Medical Association (Declaration of Helsinki). Written informed consents were obtained from all of the participants.

First, we checked whether the faithfulness evaluation for each projection pattern satisfied the results in Ref. [17], as we used observers with different characteristics from those of the previous experiments. Figure 5(a) shows the average rating value (with the standard error among observers) for each pattern. For comparison, we show the results of Ref. [17] in Fig. 5(b). As shown in the figure, except for the projection pattern S-2 (p < 0.05), there was no significant difference in each projection pattern between the previous and present data. What is different from the subjects in the previous study was that the subjects in the present study were watching the actual starry sky on a daily basis as described in Section 2. Therefore, the rating became sensitive to the faithfulness of brighter stars, and the evaluation for the pattern S-2 had might be significantly worse. Even for S-2, the average rating values of the previous and present experiments were −1.0 and − 0.3, respectively; the tendencies of negative evaluation were consistent. These results confirmed that the rating values for faithful reproduction in this experiment supported the results in Ref. [17].

The projection pattern with the highest rating of faithfulness was the pattern S-1, as shown in Fig. 5. There was a significant difference between the pattern S-1 and all other patterns. In addition, the significant difference between the Std and S-2 patterns with a negative rating value was confirmed. On the other hand, the C- and L-patterns did not have a significant difference with another pattern. These results indicate that the projection size significantly influenced the faithfulness evaluation. In contrast, the tolerance to lack of faithfulness was high regarding the color-temperature shift in the prepared patterns used in this experiment, as these patterns had no different rating values.

We can summarize the conclusions on faithful reproduction as follows. In addition, in the faithfulness evaluation for the projection pattern with a smaller star size, the rating of faithfulness improved. In contrast, the rating was remarkably negative when the pattern had stars with a larger size than that in the standard pattern. Regarding the shifts in the color temperature of the stars, the observers could distinguish differences in color but did not give negative ratings for changes in the color pattern.

In this subsection, we consider the preferred reproduction. The preference evaluation was also performed by 47 observers. Figure 6 shows the average rating value with the standard error among the 41 observers excluded outliers for each pattern. A significant difference is indicated by symbols (*: p < 0.05, **: p < 0.01). The projection pattern with the highest rating of preference was the pattern Std, as shown in Fig. 6. There was a significant difference between the patterns C-1 and C-2. In contrast, the pattern L-1 had the lowest rating of preference with a larger significant difference with all of the other patterns (p < 0.01). On the contrary, color or size change patterns or brighter patterns such as the C-3, B, S-1, S-2, and L-2 patterns had similar ratings without a significant difference with the highest-rated pattern Std. These results indicate that various observers accepted the Std pattern as the preference star reproduction, which was prepared considering the perceptual equivalence to the appearance of the actual starry sky. On the contrary, the darkness in the star reproduction sensitively negatively influenced the preference evaluation.

In this subsection, we investigate the relationship between faithfulness and preference of stars in a planetarium.

The star-image analysis in Section 3.3.1 revealed differences between the faithfulness and preference evaluations in the three patterns (L-1, L-2, and S-2) for luminance and size. We further revealed that there was a characteristic gender difference.

The gender differences of the average rating values for preference and faithfulness evaluations are shown in Figure 8. The error bars represent the standard errors among each gender (male: 33 observers, female: 14 observers) for each pattern. As shown in Fig. 8(a), the evaluation of faithfulness seems to vary between male and female observers; however, there was not a statistically significant difference, except for the projection pattern C-2 (p < 0.05). The evaluation of the pattern C-2 seems to be a convincing result, as, according to Ref. [17], the evaluation was divided into a positive and negative evaluation group. Regarding the preference evaluation, as shown in Fig. 8(b), the tendency of evaluation, positive (plus rating) and negative (minus rating), was consistent between male and female observers, and there was no statistically significant difference except for the evaluation of the pattern L-2 (p < 0.05). It is worth noting that for the female observers, the projection patterns with high brightness such as the patterns L-2 and S-2 were highly rated as preferred.

Further, within each category of male and female observers, we investigate the relationship between faithfulness and preference evaluations. Figure 9 shows the differences in rating values between the two evaluations. As shown in Fig. 9(a), for the male observers, there was no significant difference in any pattern. This suggests that the male observers expected planetariums with faithful reproduction as a preferred reproduction. In contrast, as shown in Fig. 9(b), the female observers had different impressions. The evaluations by the female observers were significantly different for two patterns (L-1 and L-2; p < 0.05). They preferred the high-luminance reproduction rather than the faithful reproduction. In other words, they expected the planetarium to reproduce the brilliant star.

Incidentally, we investigated the experience of astronomical observation by conducting a questionnaire after the experiments. We analyzed experience using observers’ data about years of astronomical observation experience, the frequency of observation, and locations where the sky was observed, etc., using several analysis methods including correlation and clustering. However, we did not find a relationship between observers’ ratings and their experience. Therefore, we concluded that the results of the gender differences did not depend on the experience in astronomical observations.

Figure 10 shows the rating distributions of faithfulness and preference evaluations for each projection pattern, for the male (Fig. 10(a)) and female (Fig. 10(b)) observers. (As a Figure A1 in Appendix, we show the gender difference of rating distributions between faithfulness and preference evaluations for each pattern.) The size of the circle represents the answer ratio for each pattern. From the upper-left panel to the bottom-right panel, the total distribution for all patterns, Std and color variation patterns, Std and luminance variation patterns, and Std and projection size variation patterns are shown, respectively. In addition, a coefficient of correlation calculated by Pearson correlation with/without the significance between the evaluations for faithfulness and preference is shown in each figure. The total distribution had a significant correlation (**: p < 0.01) for both genders; the correlation coefficient for the male observers (N = 33) was higher than that for the female observers (N = 14), despite the more than twice larger number of male observers than that of female observers. Furthermore, the results of the male observers had significant correlations for each variation pattern (p < 0.01). In contrast, a significant correlation was confirmed only for the size variation patterns in the results of the female observers (p < 0.05). In particular, the faithful and preferred reproductions were not correlated for the luminance variation pattern. These results support the results in Fig. 9; i.e., the faithful and preferred reproductions were consistent for the male observers, while for the female observers, they differed, in particular, with respect to the luminance evaluation. Figures 11(a) and (b) show the inter-participant correlation per pattern sorted in ascending order w.r.t. the correlation. As shown in the figure, in the case of a male observer, a significant correlation is confirmed between the evaluation of faithfulness and preference in all patterns. In contrast, in most patterns, there is no significant correlation, especially correlation with pattern L cannot be confirmed. These results also support the results that the faithful and preferred reproductions were consistent for the male observers, while for the female observers, they differed with respect to the luminance evaluation.