E-commerce sites are being used by millions of people to buy products in a fast and seamless manner. Users express their buying needs through search queries, and an accurate understanding of the query is necessary to return relevant items (Baeza-Yates, 2017). A crucial part of query understanding is to identify attributes inherent in the query (Li et al., 2013). For example, identifying that the query “maroon 5 dvds” has product type “dvd” helps in returning relevant items. Research on identifying query attributes in web search explores the use of semantic information (Li and Lu, 2016), user engagement (Ren et al., 2014) and external knowledge bases (Wang et al., 2015). There has been relatively less work in identifying query attributes in the e-commerce domain.
In many cases, query understanding systems have conflicting candidate attributes for a given query. In the example query, “maroon 5 dvds” the candidate attributes are the product type “dvds” the color “maroon” and the band “maroon 5”. It is not straightforward for query understanding systems to infer whether the query is referring to the band “maroon 5” or the color “maroon”. Designing algorithms to resolve conflicting query attributes can help e-commerce search systems to return more relevant items and better satisfy the buying needs of users.
This task faces several challenges. First, queries are short and contain insufficient information for systems to identify attributes. Second, knowledge bases like Wikipedia used to supplement query text in web search (Hu et al., 2009) are not focused on e-commerce domain, and can lead to insufficient and noisy information. Third, e-commerce sites have millions of users, and search algorithms have significant issues of scalability.
E-commerce search systems have access to a catalog which contains the attributes of items sold by the system. Leveraging the e-commerce catalog as a knowledge base to supplement the textual information can help to resolve conflicting query attributes. In the example query ”maroon 5 dvds”, we see from the catalog that there is a significant number of items having the product type as “dvds” and have a band attribute whereas very few of them have the color attribute. This indicates that the catalog can provide valuable information which can use to resolve conflicting query attributes. Therefore, in this paper, we propose a framework to model catalog information to better identify attributes in e-commerce queries.
Specifically, we address the following questions: How to model the catalog information to resolve conflicts in query attributes? How to evaluate the impact of the framework on on e-commerce search systems? The primary contributions of the work are
Proposing the problem of resolving conflicts in query attributes for e-commerce queries;
Proposing a framework to model attribute relations from the catalog to identify query attributes in e-commerce queries; and
Presenting evaluations of the utility of catalog information in identifying query attributes on real-world data.
The rest of the paper is organized as follows. In Section 2, we describe the proposed framework. In Section 3, we present evaluations of the framework for identifying the query attribute and its impact on ranking relevant items. We conclude in Section 4 along with possible future directions.
2. The Proposed Framework
In this section, we present our proposed framework to identify attributes in e-commerce queries. We first describe the notations used and then define the problem statement. We then use the notations to describe various aspects of the framework.
We now present the notations used in the paper. Let be the query and be the set of candidate attributes in the query. Let be the set of attributes of size without the attribute . The problem can be formally stated as follows: “Given the query , attribute , and the attribute set determine whether attribute is present in .”
We next present our framework to identify attribute values for a given query. We explore two sets of metrics to model catalog information to assist in query attribute identification. We next describe two sets of metrics along with mathematical formulations, one related to the presence of the attribute in the query, and the second related to the presence of attribute value in the query.
The first metric set computes
, the probability of an attributebeing present in the query . This is formulated as
where is the probability of attribute being present where , and is the number of items in the catalog which have value for attribute . Among such items, is the number of items having values for attribute . For a given attribute , we compute Eq 1 for all attributes , resulting in a total of feature values. According to Eq 1, if the query has attribute value , it is more likely to have attribute if more items in the catalog with also contain attribute . In the query “maroon 5 dvds”, very few items which have the product type “dvds” have values for color, and Eq 1 metric gives a lower value for , the probability of color attribute being present in the query.
The second metric computes , the probability of attribute having a value for the query . This is formulated as
where the number of items in the catalog having the value for a given attribute is . Among these items, let the number of items having value for attribute be denoted by . The score is higher if more number of items having the value for attribute also have value for attribute . We repeat this for all possible attributes for a given attribute resulting in an additional feature values. The value set for a given attribute follow a power law distribution where few values are prominent, so we employ log smoothing to make the values linearly distributed.
We integrate the scores derived from the catalog metrics into a feature set. Our metrics are scalable and hence suited for handling large-scale traffic common in an e-commerce site. We use an out of the box classifier on the feature set to determine whether or not the given query has the attribute value.
We next evaluate our framework with the help of traffic weighted random sample of 20000 queries on Walmart.com. To compare our framework, we use the baseline Dict Lookup which identifies attributes for a query by matching overalapping phrases in the query with terms in the attribute dictionary. This baseline does not address the possible conflicts that can arise between candidate attributes. For the evaluation, we take color as the attribute that has to be predicted for a query and the product type and brand as the attributes whose values are known for the query.
We design two evaluation tasks. The first task assesses the effectiveness of the framework on identifying attributes of a given query. We employ manual labeling by expert annotators for the ground truth and use Precision, Recall, and F1 as the evaluation metrics. The second task assesses the impact of the framework on the ranking relevant items for the given query. We use the orders of the query-item pair for the ground truth and nDCG@20 as the evaluation metric. The evaluation results are illustrated in Table1.
From the table, we can see that the framework is significantly better in identifying attributes of a given e-commerce query than the baseline across all the Precision, Recall and F1 metrics. The improvement in identifying query attributes is also reflected in showing better ranking results as shown by the lift in nDCG@20. The improvement in both the tasks demonstrates that the catalog can be effectively leveraged as a knowledge base to identify attributes for a given query in a better manner, and the ability of the metric to effectively capture the relevant catalog information.
4. Conclusions and Future Work
In this paper, we address the problem of identifying attributes for queries on e-commerce sites. General purpose knowledge bases used in identifying attributes for web queries are not focused on e-commerce needs. We design a framework that leverages catalog as a knowledge base to resolve conflicts in query attributes. We evaluate the framework on the set of queries from Walmart.com and demonstrate that it significantly improves results in attribute identification and ranking relevant items for e-commerce queries. Future research directions can include leveraging query catalog interactions and query sequences to design more involved metrics for attribute identification. The utility of catalog in other query understanding tasks such as query reformulation and type-ahead is also an interesting avenue for researchers to explore.
- Baeza-Yates (2017) Ricardo Baeza-Yates. 2017. Semantic Query Understanding. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’17). ACM, New York, NY, USA, 1357–1357. https://doi.org/10.1145/3077136.3096472
- Hu et al. (2009) Jian Hu, Gang Wang, Fred Lochovsky, Jian-tao Sun, and Zheng Chen. 2009. Understanding user’s query intent with wikipedia. In Proceedings of the 18th international conference on World wide web. ACM, 471–480.
- Li and Lu (2016) Hang Li and Zhengdong Lu. 2016. Deep learning for information retrieval. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 1203–1206.
- Li et al. (2013) Yanen Li, Bo-June Paul Hsu, and ChengXiang Zhai. 2013. Unsupervised identification of synonymous query intent templates for attribute intents. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge management. ACM, 2029–2038.
- Ren et al. (2014) Xiang Ren, Yujing Wang, Xiao Yu, Jun Yan, Zheng Chen, and Jiawei Han. 2014. Heterogeneous graph-based intent learning with queries, web pages and wikipedia concepts. In Proceedings of the 7th ACM international conference on Web search and data mining. ACM, 23–32.
- Wang et al. (2015) Zhongyuan Wang, Kejun Zhao, Haixun Wang, Xiaofeng Meng, and Ji-Rong Wen. 2015. Query Understanding through Knowledge-Based Conceptualization.