Understanding the behaviour of online TV users

Abstract

The amount of online video content available to us is rapidly increasing. Understanding how people are seeking and consuming this content is a prerequisite for providing good services. This paper investigates whether and how log data can be used to identify information-seeking behaviour in the context of online TV. A study was conducted where 27 participants performed given tasks on two Norwegian online TV sites. The participants were between 20 and 25 years old, and all of them were moderate or heavy users of online TV. Tasks that require both scanning and searching of information were given. Four main types of behaviour were identified in the qualitative data: goal-directed search, goal-directed metadata search followed by consumption, goal-directed search of metadata and video, and explorative behaviour. Detailed log event files were compared to self-reported data describing user’s activities (feedback collected at the end of each task and interviews) and screen captures. Our results indicate that the following four variables in the log files: number of (short navigation sequence, short video watching sequence) pairs, frequency of video search actions, percentage of time spent on sequences of navigate actions and percentage of time spent on watching videos can be used to characterise the four types of behaviour. This work extends previous research on usage of log files in describing user’s behaviour by providing simple way of characterising behaviour of online TV users. In particular, the results might be useful in supporting the personalisation of online TV services.

Appendix

Scoring reliability

The results from applying the Krippendorff KALPHA procedure on four participants were scored by two judges.

The analysis was based on [14]. The first four participants with complete log files provided the data for the reliability analysis. Two coders used an initial version of the coding scheme. In the final analysis a slight modification of this scheme was used. Each participant completed a total of 10 tasks, including warm-up tasks. The coding scheme had a total of 48 categories. Each category could be coded zero, one or several times within each task. The basic data were the number of times a specific category was used within a task. Each participant had total of 480 scores. Analysing the four participants together gave a data set of 1920 values, scored by the two coders. As pointed out by Hayes and Krippendorff [14], “In its two-observer interval data version, alpha equals Pearson intraclass-correlation coefficient”.

In the analysis the measurement level was set at intervals (level set to 3 in the analysis). If one of the judges had used the category zero times within the task, that unit was left out of the analysis. Bootstraps were set to 1000.

Krippendorff alpha was 0.8727. As pointed out by Krippendorff [19] pp. 241–243, “social scientists commonly rely on data with reliabilities α ≥ 0.800, consider data with 0.800 > α ≥ 0.667 only to draw tentative conclusions, and discard data whose agreement measures α < 0.667” (Table 7).

Table 8 lists the results from the questionnaire the participants completed before the test started. Tables 9 and 10 show the differences between the participants with and without valid log data. The nonparametric “median test” was chosen since most of the data distributions were rather skewed. As expected, there were no significant differences between the two groups, although there was a slight tendency that, proportionally, more females than males had valid log data. In Table 9, one can see, for example, that all the participants with valid data went on Facebook daily, while only one of those without valid data did not.

Table 7 Sequence elements

Table 8 Frequency of demographic characteristics and Web usage

Table 9 Category of participation, compared to questionnaire responses

Table 10 Median test of significance for questionnaire answers and category of participation—With valid log data versus without valid log data