Cost-sensitive Learning of Deep Semantic Models for Sponsored Ad Retrieval
share
This paper formulates the problem of learning a neural semantic model for IR as a cost-sensitive learning problem. Current semantic models trained on click-through data treat all historical clicked query-document pairs as equivalent. It argues that this approach is sub-optimal because of the noisy and long tail nature of click-through data (especially for sponsored search). It proposes a cost-sensitive (weighted) variant of the state-of-the-art convolutional latent semantic model (CLSM). We explore costing (weighing) strategies for improving the model. It also shows that weighing the pair-wise loss of CLSM makes the upper bound on NDCG tighter. Experimental evaluation is done on Bing sponsored search and Amazon product recommendation. First, the proposed weighted model is trained on query-ad pairs from Bing sponsored search click logs. Online A/B testing on live search engine traffic shows 11.8% higher click-through rate and 8.2% lower bounce rate as compared to the unweighted model. Second, the weighted model trained on amazon co-purchased product recommendation dataset showed improvement of 0.27 at NDCG@1 and 0.25 at NDCG@3 as compared to the unweighted model.
READ FULL TEXT