Evaluating books finding tools on social media: A case study of aNobii
Introduction
The growing popularity of social media has captured the public imagination and attracted attention from academic and industry researchers. For information scientists, the study of social media carries a special sense of urgency as they have rapidly become the main venues where information is consumed and exchanged. Socially-enabled navigational tools embedded in social media afford novel modes of access beyond those of traditional bibliographic means. For example, on aNobii, a social networking site for avid readers, users can now explore previously unknown items through socially-enabled navigational tools such as browsing friends’ and system-generated similar bookshelves. These book-finding tools can greatly complement traditional subject access devices in two ways. Firstly, they are particularly effective when users’ information needs are ill-defined and inexpressible, such as is often the case in searching for leisure reading. Secondly, they greatly increase opportunities for serendipitous finds through associative browsing. Fellow readers’ comments and ratings can also provide important judgment cues for book quality.
The newly emerging information environment on social media presents a fertile ground for information behavior research as new tools often engender new user information behaviors. Users now can rely on socially-enabled cues such as recommendations and reviews by others to come to know items of interest. Yet so far little research has been done to study users’ information behaviors in such a cue-rich environment, nor has there been a consensus on how to evaluate the tools embedded in these sites. The exploratory, as opposed to keyword-matching, mode of access on social media also presents a great challenge to evaluation methodology. It remains unclear what evaluative criteria, other than relevance-based precision and recall measures, can be used to validate the benefits provided by these exploration based tools. Other than search results, the user experience might be an important aspect of value provided by these tools. For example, researchers have pointed out exploratory systems can prove beneficial to support an interactive learning process beyond search results (Marchionini, 2006, White et al., 2006, White et al., 2008).
Furthermore, what really matters in users’ searching of information items for leisure on these sites is users’ preference rather than topic relevance (Koolen, Kamps, & Kazai, 2012). While in marketing research users’ responses to customized offers have been shown to be influenced by their knowledge and involvement with the products (Franke et al., 2009, Kwon et al., 2009, Shen and Ball, 2011, Simonson, 2005). Yet little has been investigated, from a system evaluation perspective, on how different user traits might influence their seeking behaviors and system performance with these socially-based recommendation tools. In the book finding environment on social media, users are constantly exposed to novel and sometimes serendipitous items. Thus, their willingness and ability to explore items might turn out to be an important factor on the effectiveness of these tools. One can imagine that readers with a more refined preference mightreact differently than more casual readers. It might also be relatively more difficult to recommend novel and potentially interesting items for well-read readers and more resourceful readers who rely on their own information resources for book finding. Among cultural product consumers, a distinction is often made between sophisticated and casual consumers (Caves, 2002). When examining online DVD rental data, Elberse (2008) found that consumers who have ever chosen obscure products tended to consume a much wider range of alternatives, including popular items, while those who knew of few alternatives tended to stick with more popular offerings. It is reasonable to hypothesize that users’ knowledge and involvement with cultural products influence their response to customized recommendations. Therefore besides testing various evaluation criteria in this study, we also wish to explore how different aspects of users’ preference might influence user behaviors and tool effectiveness.
In the following sections, we will begin with a discussion of the impact of socially-based book-finding tools on users’ access to readings for leisure, followed by elucidating the procedures and results of the experiment.
Section snippets
Access to readings for pleasure
Community-based navigational tools can be especially effective where keyword-based subject access is less successful (Koolen et al., 2012). An area where subject access has been found to be inadequate is access to imaginary work. With imaginary works, what is sought is the emotive reading experience that the work evokes rather than the subject it addresses. The difficulty of subject access to imaginary works lies in the fact that the relationship between the features in the work and the types
Research questions
The aims of the study are twofold: Firstly, to explore possible criteria for evaluating the social navigational tools. A set of measures designed to capture various aspects of the benefits provided by the tools was proposed. An experiment was then conducted using a modified interface of aNobii so that we could compare performance of these tools against the traditional author browsing function on these measures. Secondly, the study examines how three aspects of preference, “preference insight”,
The testing site
The study set out to explore potential criteria by which the social navigational tools on social media can be evaluated. The online book sharing website aNobii was chosen as the test site for our study, for a couple of reasons. Firstly, it has attracted a relatively large user community in Mandarin in Taiwan (where the study was conducted) among all the book-sharing platforms. Secondly, it also provides a wide range of social navigational tools that enable us to draw rich user-interaction data.
Descriptive statistics
Table 3 provides the descriptive statistics of the participants’ gender, time since joining aNobii, and frequency of use. Over 82 percent of the participants have joined for more than one year. The numbers of books owned in the “bookshelf” ranged greatly from 81 to 2528, with a mean of 431.65 and SD of 495.30; the numbers of friends ranged from 5 to 52, with a mean of 15.6 and SD of 11.53. A significant correlation was found between the number of books owned and total friends, r = .31, p < .05 (see
Discussion
Our results showed that while the similar bookshelf did consistently the poorest, the other two book-finding tools, author search and friends’ bookshelves exchanged leads in different measures, depending on the underlying criteria these measures represented. In the search experience category, “friends’ bookshelves” did better in all but one measure, “preference matching”. The advantage of the browsing known- author approach in preference matching was also reflected in users’ visiting and saving
Conclusion
The growing availability of social navigational tools on social media presents a challenge to evaluative methodology. The first main contribution of our research is to propose and test a set of subjective and objective evaluative measures by which the benefits of these tools can be validated and compared. Other than the usual self-report user experience and result based measures, the “choice set” model from retailing was used to provide a framework by which navigational effectiveness can be
References (40)
- et al.
Preference construction and preference stability: Putting the pillow to rest
Journal of Consumer Psychology
(2008) - et al.
A collaborative filtering similarity measure based on singularities
Information Processing & Management
(2012) Structure of vacation destination choice sets
Annals of Tourism Research
(1992)Supporting social navigation on the World Wide Web
International Journal of Human-Computer Studies
(1997)- et al.
Influences of customer preference development on the effectiveness of recommendation strategies
Electronic Commerce Research and Applications
(2009) - et al.
The influence of goal-directed and experiential activities on online flow experiences
Journal of Consumer Psychology
(2003) - et al.
Model-driven formative evaluation of exploratory search: A study under a sensemaking framework
Information Processing & Management
(2008) - et al.
Online customers’ cognitive differences and their impact on the success of recommendation agents
Information & Management
(2010) - et al.
The dimensionality and correlates of flow in human-computer interactions
Computers in Human Behavior
(1993) - et al.
Evaluating exploratory search systems
Information Processing & Management
(2008)
Creative industries: Contracts between art and commerce
The role of involvement in attention and comprehension processes
Journal of Consumer Research
Testing the value of customization: When do customers really prefer products tailored to their preferences?
Journal of Marketing
Audience involvement in advertising: Four levels
Journal of Consumer Research
Evaluating collaborative filtering recommender systems
ACM Transactions on Information Systems
The effect of cultural orientation on consumer responses to personalization
Marketing Science
Cited by (14)
Exploring the impacts of a recommendation system on an e-platform based on consumers’ online behavioral data
2024, Information and ManagementFinding fiction: Search moves and success in two online catalogs
2016, Library and Information Science ResearchCitation Excerpt :The strongest predictor of successful book selection was the duration of visits to the author/title information, suggesting that the content description did not play a crucial role in fiction selection. In fiction searching, the concept of social browsing (Peters, 2011) refers to the ability to re-orientate browsing by social navigational tools (Tang, Sie, & Ting, 2014). Tang et al. conducted a user study of aNobii, an online book sharing website.
Exploratory search in information systems: a systematic review
2024, Electronic LibraryPreference diversity and openness to novelty: Scales construction from the perspective of movie recommendation
2022, Journal of the Association for Information Science and TechnologyDigital humanities and digital social reading
2021, Digital Scholarship in the HumanitiesMusic discovery and revisiting behaviors of individuals with different preference characteristics: An experience sampling approach
2020, Journal of the Association for Information Science and Technology