Better Late Than Never but Never Late Is Better: Towards Reducing the Answer Response Time to Questions in an Online Learning Community

Abstract

Professionals increasingly turn to online learning communities (OLCs) such as Stack Overflow (SO) to get help with their questions. It is important that the help is appropriate to the learning needs of the professional and is received in a timely fashion. However, we observed in SO a rise in the proportion of questions either answered late or not answered at all, from 5% in 2009 to 23% in 2016. There is clearly a need to be able to quickly find appropriate answerers for the questions asked by users. Our research goal is thus to find techniques that allow us to predict from SO data (using only information available at the time the question was asked) the actual answerers who provided the best answers and the most timely answers to users’ questions. Such techniques could then be deployed proactively at the time a question is asked to recommend an appropriate answerer. We used a variety of tag-based, response-based, and hybrid approaches in making these predictions. Comparing the approaches, we achieved success rates that varied from a low of .88% to a high of 89.64%, with the hybrid approaches being the best. We also explored the effect of excluding from the pool of possible answerers those users, who had already answered a question “recently”, with “recent” varying from 15 min up to 12 h, so as to have well rested helpers. We still achieved reasonable success rates at least for smaller exclusion periods of up to an hour, although naturally not as good as the time exclusion grew longer. We believe our work shows promise for allowing us to predict prospective answerers for questions who are not overworked, hence reducing the number of questions that would otherwise be answered late or not answered at all.