DeepAI AI Chat
Log In Sign Up

Handling Long-Tail Queries with Slice-Aware Conversational Systems

by   Cheng Wang, et al.

We have been witnessing the usefulness of conversational AI systems such as Siri and Alexa, directly impacting our daily lives. These systems normally rely on machine learning models evolving over time to provide quality user experience. However, the development and improvement of the models are challenging because they need to support both high (head) and low (tail) usage scenarios, requiring fine-grained modeling strategies for specific data subsets or slices. In this paper, we explore the recent concept of slice-based learning (SBL) (Chen et al., 2019) to improve our baseline conversational skill routing system on the tail yet critical query traffic. We first define a set of labeling functions to generate weak supervision data for the tail intents. We then extend the baseline model towards a slice-aware architecture, which monitors and improves the model performance on the selected tail intents. Applied to de-identified live traffic from a commercial conversational AI system, our experiments show that the slice-aware model is beneficial in improving model performance for the tail intents while maintaining the overall performance.


page 1

page 2

page 3

page 4


Learning Slice-Aware Representations with Mixture of Attentions

Real-world machine learning systems are achieving remarkable performance...

Slice-based Learning: A Programming Model for Residual Learning in Critical Data Slices

In real-world machine learning applications, data subsets correspond to ...

Slice Finder: Automated Data Sclicing for Model Validation

As machine learning (ML) systems become democratized, it becomes increas...

Self-Aware Feedback-Based Self-Learning in Large-Scale Conversational AI

Self-learning paradigms in large-scale conversational AI agents tend to ...

Slice-Aware Neural Ranking

Understanding when and why neural ranking models fail for an IR task via...

Beyond Accuracy: Cost-Aware Data Representation Exploration for Network Traffic Model Performance

In this paper, we explore how different representations of network traff...

On Exposing the Challenging Long Tail in Future Prediction of Traffic Actors

Predicting the states of dynamic traffic actors into the future is impor...