Algorithms for the Greater Good! On Mental Modeling and Acceptable Symbiosis in Human-AI Collaboration

01/30/2018
by   Tathagata Chakraborti, et al.
0

Effective collaboration between humans and AI-based systems requires effective modeling of the human in the loop, both in terms of the mental state as well as the physical capabilities of the latter. However, these models can also open up pathways for manipulating and exploiting the human in the hopes of achieving some greater good, especially when the intent or values of the AI and the human are not aligned or when they have an asymmetrical relationship with respect to knowledge or computation power. In fact, such behavior does not necessarily require any malicious intent but can rather be borne out of cooperative scenarios. It is also beyond simple misinterpretation of intents, as in the case of value alignment problems, and thus can be effectively engineered if desired. Such techniques already exist and pose several unresolved ethical and moral questions with regards to the design of autonomy. In this paper, we illustrate some of these issues in a teaming scenario and investigate how they are perceived by participants in a thought experiment.

READ FULL TEXT

page 3

page 4

research
10/15/2019

Challenges of Human-Aware AI Systems

From its inception, AI has had a rather ambivalent relationship to human...
research
09/05/2023

Designing Interfaces for Human-Computer Communication: An On-Going Collection of Considerations

While we do not always use words, communicating what we want to an AI is...
research
04/15/2022

Identifying Ethical Issues in AI Partners in Human-AI Co-Creation

Human-AI co-creativity involves humans and AI collaborating on a shared ...
research
03/28/2022

Human-AI Collaboration Enables More Empathic Conversations in Text-based Peer-to-Peer Mental Health Support

Advances in artificial intelligence (AI) are enabling systems that augme...
research
01/19/2022

When Is It Acceptable to Break the Rules? Knowledge Representation of Moral Judgement Based on Empirical Data

One of the most remarkable things about the human moral mind is its flex...
research
05/15/2020

Exploring Crowd Co-creation Scenarios for Sketches

As a first step towards studying the ability of human crowds and machine...
research
04/12/2021

Building Mental Models through Preview of Autopilot Behaviors

Effective human-vehicle collaboration requires an appropriate un-derstan...

Please sign up or login with your details

Forgot password? Click here to reset