Evaluation of algorithms for combining independent data sets in a human performance expert system

As part of an ongoing program to develop a Computer Aided Engineering (CAE) system for human factors engineers, a Human Performance Expert System, Human, was designed. The system contains a large database of human-performance equations derived from human performance research reported in the open literature. Human accesses these data to predict task performance times, task completion probabilities, and error rates. A problem was encountered when multiple independent data sets were relevant to one task. For example, a designer is interested in the effects of luminance and font size on a number of reading errors. Two data sets exist in the literature: one examining the effects of luminance, the other, font size. The data in the two sets were collected at different locations with different subjects, and at different times in history. How can the two data sets best be combined to address the designer's problems?

On the basis of an extensive review of the human performance literature and statistical procedures, four combining algorithms were developed. These four algorithms were tested in two steps. In step one, two reaction-time experiments were conducted: one to evaluate the effect of the number of displays being monitored. The four algorithms were used on the data from these two experiments to predict reaction time in the situation where all three independent variables are manipulated simultaneously. In step two of the test procedure, a third experiment was conducted. Subjects who had not participated in either Experiment 1 or 2 performed a reaction-time task under the combined effects of all three independent variables. The predictions made from step one were compared to the actual empirical data collected in Experiment 3. The best predictor of the mean in Experiment 3 was an unweighted average of the means in Experiments 1 and 2; the best predictor of the standard deviation in Experiment 3 was an unweighted average of the standard deviations, (S.D.s) in Experiments 1 and 2. Based on these results, Human uses an average of the means to combine the results from multiple independent data sets.