Objective and Subjective Responsibility of a Control-Room Worker
share
When working with AI and advanced automation, human responsibility for outcomes becomes equivocal. We applied a newly developed responsibility quantification model (ResQu) to the real world setting of a control room in a dairy factory to calculate workers' objective responsibility in a common fault scenario. We compared the results to the subjective assessments made by different functions in the diary. The capabilities of the automation greatly exceeded those of the human, and the optimal operator should have fully complied with the indications of the automation. Thus, in this case, the operator had no unique contribution, and the objective causal human responsibility was zero. However, outside observers, such as managers, tended to assign much higher responsibility to the operator, in a manner that resembled aspects of the "fundamental attribution error". This, in turn, may lead to unjustifiably holding operators responsible for adverse outcomes in situations in which they rightly trusted the automation, and acted accordingly. We demonstrate the use of the ResQu model for the analysis of human causal responsibility in intelligent systems. The model can help calibrate exogenous subjective responsibility attributions, aid system design, and guide policy and legal decisions.
READ FULL TEXT