DeepAI AI Chat
Log In Sign Up

High-quality Conversational Systems

by   Samuel Ackerman, et al.

Conversational systems or chatbots are an example of AI-Infused Applications (AIIA). Chatbots are especially important as they are often the first interaction of clients with a business and are the entry point of a business into the AI (Artificial Intelligence) world. The quality of the chatbot is, therefore, key. However, as is the case in general with AIIAs, it is especially challenging to assess and control the quality of chatbot systems. Beyond the inherent statistical nature of these systems, where occasional failure is acceptable, we identify two major challenges. The first is to release an initial system that is of sufficient quality such that humans will interact with it. The second is to maintain the quality, enhance its capabilities, improve it and make necessary adjustments based on changing user requests or drift. These challenges exist because it is impossible to predict the real distribution of user requests and the natural language they will use to express these requests. Moreover, any empirical distribution of requests is likely to change over time. This may be due to periodicity, changing usage, and drift of topics. We provide a methodology and set of technologies to address these challenges and to provide automated assistance through a human-in-the-loop approach. We notice that it is crucial to connect between the different phases in the lifecycle of the chatbot development and to make sure it provides its expected business value. For example, that it frees human agents to deal with tasks other than answering human users. Our methodology and technologies apply during chatbot training in the pre-production phase, through to chatbot usage in the field in the post-production phase. They implement the `test first' paradigm by assisting in agile design, and support continuous integration through actionable insights.


page 9

page 10

page 12


Perspectives for Evaluating Conversational AI

Conversational AI systems are becoming famous in day to day lives. In th...

AIQ: Measuring Intelligence of Business AI Software

Focusing on Business AI, this article introduces the AIQ quadrant that e...

Explainable AI for System Failures: Generating Explanations that Improve Human Assistance in Fault Recovery

With the growing capabilities of intelligent systems, the integration of...

An Automated Testing Framework for Conversational Agents

Conversational agents are systems with a conversational interface that a...

Enabling Value Sensitive AI Systems through Participatory Design Fictions

Two general routes have been followed to develop artificial agents that ...

Look Before You Leap! Designing a Human-Centered AI System for Change Risk Assessment

Reducing the number of failures in a production system is one of the mos...

Insightful Assistant: AI-compatible Operation Graph Representations for Enhancing Industrial Conversational Agents

Advances in voice-controlled assistants paved the way into the consumer ...