Modeling problem difficulty and expertise in stack overflow

Abstract

Supporting effective community engagement and crowdsourcing is becoming a ‘must-have’ capability in various application domains. Examples include involving employee-groups for generating and sharing practical knowledge within organizations (e.g., Xeroxs Eureka system), engaging the voice of external customers (e.g., mystarbucksidea), and “crowdsourced” helpdesk services (e.g., recent investigations of enterprise crowdsourcing). The design of socio-technical systems that incorporate help-based communities needs to enable synergy among different types of actors: a crowd of experts or users interacting with the system to request help (e.g., helpdesk operator) or providing help (an employee or customer with a question or a problem), as well as non-human agents (e.g., machine learning algorithms operating on the interaction data). In order to inform the design of such systems we have studied a successful help-based community called Stack Overflow. Specifically, we focus on analyzing and modeling how complex problems are being handled in this community and dispatched to different actors. In this paper we describe how the difficulty of the problems can be measured and related to the measures of expertise of individuals and in social networks of the people involved. We draw implications for the design of future systems seeking to leverage help-based, ‘enthusiast’ communities.