New assessment criteria for query suggestion

Query suggestion is a useful tool to help users express their information needs by supplying alternative queries. When evaluating the effectiveness of query suggestion algorithms, many previous studies focus on measuring whether a suggestion query is relevant or not to the input query. This assessment criterion is too simple to describe users' requirements. In this paper, we introduce two scenarios of query suggestion. The first scenario represents cases where the search result of the input query is unsatisfactory. The second scenario represents cases where the search result is satisfactory but the user may be looking for alternative solutions. Based on the two scenarios, we propose two assessment criteria. Our labeling results indicate that the new assessment criteria provide finer distinctions among query suggestions than the traditional relevance-based criterion.