Worker Skill Estimation from Crowdsourced Mutual Assessments

Abstract

Current approaches for estimating skill levels of workforce either do not take into account the expertise of the recommender, or require intricate and expensive processes. In this paper, we propose a crowdsourcing algorithm for worker skill estimation based on mutual assessments. We propose a customized version of PageRank algorithm wherein we specifically considered the expertise of the person who made assessments. By implementing our algorithm on 15 real-world datasets from organizations and companies of varying sizes and domains and by using leave-one-out cross validation, we find that the results are highly correlated with the ground truth in datasets.