MATE robots simplifying my work: benefits and socio-ethical implications

12/20/2017
by   Valeria Villani, et al.
RWTH Aachen University
0

With the increasing complexity of modern industrial automatic and robotic systems, an increasing burden is put on the operators, who are requested to supervise and interact with very complex systems, typically under challenging and stressful conditions. To overcome this issue, it is necessary to adopt a responsible approach based on the anthropocentric design methodology, such that machines adapt to the humans capabilities, and not vice versa. Moving along these lines, in this paper we consider an integrated methodological design approach, which we call MATE, consisting in devising complex automatic or robotic solutions that measure current operator's status, adapting the interaction accordingly, and providing her/him with proper training to improve the interaction and learn lacking skills and expertise. Accordingly, a MATE system is intended to be easily usable for all users, thus meeting the principles of inclusive design. Using such a MATE system gives rise to several ethical and social implications, which are discussed in this paper. Additionally, a discussion about which factors in the organization of companies are critical with respect to the introduction of a MATE system is presented.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

page 4

06/07/2018

Methodological Approach for the Evaluation of an Adaptive and Assistive Human-Machine System

With the increasing complexity of modern industrial automatic and roboti...
10/11/2021

Ethical Assurance: A practical approach to the responsible design, development, and deployment of data-driven technologies

This article offers several contributions to the interdisciplinary proje...
04/15/2021

Ethical Challenges in the Human-Robot Interaction Field

The aim of this extended abstract is to introduce five examples of ethic...
06/08/2018

An Industrial Social Network for Sharing Knowledge Among Operators

Due to the increasing complexity of modern automatic machines typically ...
07/09/2020

A Reference Software Architecture for Social Robots

Social Robotics poses tough challenges to software designers who are req...
09/14/2020

Implications of ageing for the design of cognitive interaction systems

We are living longer in times of the biggest technological revolution hu...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Anthropocentric Analysis of Requirements

Fig. 2: The process for defining technical, ELSI, and roboethics requirements for the design of human-centered complex automatic systems. MEESTAR: a model for the ethical evaluation of sociotechnical arrangements.

As with any complex system, the development of a MATE system requires defining specifications that provide a description of the main characteristics that the system must fulfill. Following a standard system engineering approach [18], these specifications are derived from a set of requirements that describe the desired behavior of the system from the user’s perspective.

While system specifications depend on the specific application under consideration, the requirements can be defined from a universal point of view (for MATE systems, in general). Derived from an anthropocentric approach, these requirements focus on the user’s needs.

Technical requirements were first considered, but they were complemented by additional necessary requirements regarding social and ethical implications of the system. Figure 2 demonstrates how the different classes of requirements are derived. All of the requirements, which have general validity, contribute to defining the technical specifications of a given system. Table I summarizes the proposed design recommendations in terms of technical requirements; requirements for the ethical, social, and legal implications (ELSI); and roboethics.

TABLE I: Design recommendations for a MATE system: technical and ELSI requirements for the inclusion of vulnerable users and principles proposed by debate on roboethics [15].

I-a Technical Requirements

To obtain the design requirements, we formed a thorough understanding of the users and analyzed their needs. We focused on three specific industrial case studies that represent a broad area of interest in Europe:

  • [Use case 1:]

  • use case 1: machinery for woodworking, typically used in small companies run by elderly artisans

  • use case 2: robotic solutions to automatize the assembly of appliances, currently done manually

  • use case 3: automatic bottling machines used in industrial plants.

Such use cases have been analyzed in detail, with specific attention given to precise working scenarios and those tasks involving interaction between the human operator and the machine [19]. To understand the limitations and pitfalls that may reduce an operator’s work performance, his or her current interaction with standard automation and robotics systems was observed [19]. Particular attention was paid to vulnerable users, since they are the most directly affected by the complexity of this interaction. For the purpose of these use cases, we consider vulnerable users not only those who have physical or cognitive impairments, but also those who are elderly and those with less education who may not be familiar with advanced technology and computerized solutions. These users are considered vulnerable because they are more likely to lose their jobs and they are less likely to be retrained or reemployed. This results from their limited ability to effectively utilize complex, modern computer-aided manufacturing equipment, as well as from impairments that restrict the performance of physically demanding tasks.

A list of users’ needs, with respect to current human-machine systems, is summarized in Table II. These needs highlight the fact that current interaction modality involves significant effort from users, particularly from vulnerable ones, which often results in additional stress for workers. A simplification of this interaction, by means of guidance in procedural tasks and easy-to-use interaction modes, must be achieved. The first set of requirements derived from these users’ needs is listed in the left column of Table I and translates users’ needs to technical hints for design to improve traditional approaches to interaction.

Note that these technical requirements are applicable to any inclusive interaction system and do not depend on the previously described use cases, though they have been derived from specific users’ needs that are related to the use cases described in this article. Additionally, these users’ needs have been developed as additional requirements to overcome the limitations of traditional systems, and they are not the only technical requirements to consider in the system’s design (e.g., task-specific requirements and safety requirements).

I-B ELSI Requirements

In addition to the technical requirements obtained from the use cases, other design recommendations are derived from the analysis of ELSIs related to the use of an adaptive MATE human–machine system. This analysis helps to define the ELSI requirements. To develop an ELSI concept related to the design of smart interaction systems for automated production machines, we have combined a model for the ethical evaluation of sociotechnical arrangements (MEESTAR) [20], which considers ethical problems, and the context-specific legal issues for occupational systems, as well as societal needs of the vulnerable users [19].

The MEESTAR approach is a three-dimensional (3-D) evaluation tool that guides users to ethically reflect upon and form judgments relative to the use of the appropriate systems. MEESTAR requires the system to cause little or no harm to the user and attempts to identify ethically problematic effects in a structured way to develop appropriate solutions. As shown in Figure 3, it divides this analysis into three main components and, when considered jointly, allows for a comprehensive ethical evaluation of a sociotechnical arrangement. The first dimension looks at ethical core values, i.e., care, autonomy, safety, justice, privacy, participation, and self-conception. The second dimension considers three levels of observation, i.e., individual, organizational, and societal, and it includes the perspectives of those who use the system and the people in close proximity to the user. The third dimension identifies four stages of ethical sensitivity, ranging from “completely harmless” to “should be opposed from an ethical viewpoint.” A detailed description of the MEESTAR approach is found in [20].

With regard to the legal implications, we consider data protection, safety, and health at work, as addressed by the European Union (EU) regulations, i.e., the Machinery Directive 2006/42/EC, the Council Directive 89/391/EEC, and the General Data Protection Regulation 2016/679. For the previously discussed vulnerable users, whose presumed limited perception skills and cognitive and motor skills would otherwise hinder their effectiveness, specific social requirements should be implemented to help ensure their equal treatment and integration into the work process.

Fig. 3: A 3-D MEESTAR approach for the assessment of ethical issues related to the use of technology. x axis: dimensions of ethical evaluation; y axis: stages of ethical evaluation; z axis: levels of ethical evaluation [20].

I-C Roboethics Requirements

The ELSI requirements previously presented do not provide a complete analysis of the design recommendations from an ethical point of view. Since we are dealing with vulnerable users and considering a system that measures human capabilities and performance, ethical issues are of specific concern and must be dealt with in detail. Additional ethical principles for the design, production, and use of robots are outlined in the framework of roboethics [15]. Although they are quite general, such roboethics requirements are well-suited to the MATE methodology and suggest further issues that are not covered by ELSI analysis. The requirements proposed by roboethics are reported in the last column of Table I. Two roboethics requirements (ROBETH-R1 and ROBETH-R5) overlap with ELSI principles (ELSI-R5 and ELSI-R2, respectively) and focus on the importance of the system not causing harm to the operator, as well as the use of anonymous personal data.

Roboethics acknowledges that the worker should always maintain control of the interaction (ROBETH-R2 and ROBETH-R3), dismissing it whenever he or she does not feel comfortable with the system. Thus, the user of a MATE system should have complete autonomy; for example, if a user experiences reluctance interacting with a MATE system, he or she should be free to use standard interaction approaches, though this choice would result in a decreased level of performance.

Roboethics requirement ROBETH-R6 focuses on one of the biggest drawbacks of using MATE systems in industrial applications: the risk of stigmatization that could arise from the use of a system that measures a worker’s skills and performance. This may be a sensitive issue for the worker, who may feel embarrassment from the assessment, and for his or her supervisor or employer. Although this risk is partly mitigated by precautions put in place for protecting data, care must be taken when considering MATE approaches.

While social relationships in the workplace can be undermined by the introduction of robots, the MATE system only minimally aggravates this situation. Furthermore, the only decline that is observed in social ties among colleagues is due to less support being necessary for the user of a MATE system, since the system acts as support for the user. However, the social value of these exchanges of information is debatable, since they mostly give rise to competition and stigmatization. Additionally, the use of a MATE system is expected to increase an operator’s confidence in the use of the machine or robot, thus making him or her less ashamed and more prone to interact with colleagues.

The roboethics ROBETH-R8 requirement demonstrates that, for professional users, difficulties with accessing robotic technologies may stem from a lack of confidence or knowledge of robotics [15]. The MATE approach aims at reducing these barriers by relying on an inclusive design. The inclusive design requirement is intrinsically satisfied with the implementation of a MATE system.

ROBETH-R9, the final roboethics requirement, has a marginal application to MATE systems. This requirement refers to the fact that, by using advanced robotic solutions, humans might lose the perception of their actual abilities and feel empowered by attributing a robot’s skills to themselves [15]. The first step of a MATE system is the measurement of a human’s skills, and, although stigmatization may be prevented, operators will unavoidably be faced with their own limitations measured by the system, as discussed in roboethics requirement ELSI-R6.

1) Inclusion of all users
1.1) The system should be effectively usable by inexperienced operators
1.2) The system should be effectively usable by operators with different age
1.3) The system should be effectively usable by operators with different level of work experience or education
1.4) The system should be effectively usable by operators with physical impairments
2) User-oriented organization of information
2.1) Procedures should adapt to the operator’s skills
2.2) The system should provide guided procedures for ordinary operations
2.3) The system should guide the operator according to common practice solutions
2.4) Specific prior training and studying the manual should not be necessary
2.5) Operations should be performed in the correct sequence, according to the manual
2.6) The system should suggest the operator what parameters need to be changed, based on the desired result
3) Prioritization of human factors
3.1) The system should be comfortable for all the users
3.2) The stress level during the use of the system should be low
3.3) The intervention of supervisors to assist the operators should be avoided
3.4) Operators should feel confident when using the system
4) Enhancement of operator’s performance
4.1) The number of errors should be reduced
4.2) The execution time should be improved
4.3) The correct operational mode and the correct value for critical parameters should be automatically selected
4.4) The choice of wrong options should be prevented
4.5) The HMI should depict the actual equipment and state of the machine
5) Advanced technological solutions
5.1) Hands-free interaction should be possible
5.2) Portable interfaces should be available, to guide the operators in the working area
TABLE II: Users needs derived from the analysis of the limitations on the current human–machine systems.

Ii Impact of the MATE Approach in the Organization of a Company

The practical implementation of the requirements in Table I and the impact of a MATE system on the current organization of a company are examined in this section. Satisfying the general recommendations previously discussed requires that the design of new robotic or automated solutions follows the technical requirements and the principles of inclusive design; these may be applied differently depending on the specific characteristics of any given usage. Conversely, it is necessary for some concrete actions to be taken at a broader level that consider a company’s organizational structure and goals. An infrastructure is necessary to support the introduction of a MATE system, as well as to preserve the operator’s interests.

To address this issue, we explored how the ELSI analysis affects a company’s organization. In particular, the intersection between the MEESTAR approach and the legal and social requirements leads to the following ELSI dimensions, which are discussed in detail in [19]:

  • occupational health

  • occupational safety

  • data protection

  • ergonomic workplace design

  • equal opportunities

  • reintegration.

These factors represent the main organizational dimensions that are affected by the introduction of a MATE system in an industrial setting.

We first analyzed how companies are currently organized with respect to these factors, aiming to understand the corresponding successful measures previously taken and companies’ predisposition to invest in these soft skills. We then estimated how a MATE system would affect these dimensions. The related potential for improvement and the associated risks following the introduction of a MATE interaction approach are discussed in

[19].

Information was gathered from a questionnaire that was designed to investigate the appropriateness of identified ELSI factors of MATE systems that were accessible to user groups with special needs and requirements. The questionnaire was organized in two parts: the first part investigated the status quo of the company with regard to the ELSI dimensions identified; the second part estimated the impact of a MATE system on these dimensions. The questionnaire was distributed to 13 company managers from the following diverse sectors: information technology and software development (two companies located in Germany), technology transfer (two located in Italy), industrial automation (four in Italy, one in Greece), white goods (one in Italy), packaging and bottling (two in Italy), and machine manufacturing (one in Germany).

Ii-a Status Quo of Companies with Regard to ELSI Dimensions

Of those that completed the questionnaire, Figure 4 illustrates an overview of the companies that currently include a management system for each of the ELSI dimensions. The data show that, among participants, the most commonly implemented management systems are occupational safety and health management systems and data protection divisions. It was demonstrated that, on average, companies used to employ more people in these divisions.

Additionally, Figure 5 shows the mean value of the estimated overall relevance of each dimension for the companies involved in the questionnaire. Relevance was rated on a scale from 1 (very negative) to 10 (very positive). Occupational safety, with an average relevance of , data protection at , and occupational health at , were found to be most relevant. The relevance of the other factors was rated at below mean: for ergonomic workplace design, for equal opportunities, and for reintegration. Note that, since some companies do not implement a management system for some of the ELSI dimensions examined, it was not possible to assess the corresponding relevance for all companies. Accordingly, the results in Figure 5 were averaged based on a different number of study participants. Furthermore, answers provided in the questionnaires indicated that successful measures previously implemented by the companies with respect to the ELSI factors included periodic health monitoring, psychological support, internal physiotherapists, fitness programs, reintegration of handicapped people, and reintegration after long-term illness.

Fig. 4: The number of companies currently using a management system for each of the ELSI dimensions.
Fig. 5: The mean of the estimated overall relevance of ELSI dimensions by companies involved in the questionnaire. The number of averaged answers is reported above each bar.

Ii-B MATE Impact on ELSI Dimensions

The second part of the questionnaire considered a working scenario where affective computing is applied to an industrial human–machine system, thereby measuring an operator’s mental workload, stress, and induced anxiety by recording certain physiological signals. The questionnaire included specific questions regarding the following MATE scenario: “The working machines are equipped with sensors that are able to track the strain of a working person by real-time measurement of his or her physiological parameters, e.g., heart rate and blood pressure. If the measured strain indicators are too high, the system adapts to the situation resulting in a lower stress level.

Participants were first asked whether the potential for improvement or risks in measuring strain of a working person as described in the aforementioned scenario could be found (the results are summarized in Figure 6). Most subjects reported the potential for improvement by means of the system in question: occupational health, 12 out of 13; ergonomic workplace design, 11 out of 13; and occupational safety, 9 out of 13. Occupational health, however, was also rated with the highest potential risks (six mentions), while five subjects out of 11 mentioned a potential risk with respect to data protection, ergonomic workplace design, and equal opportunities (two did not answer the corresponding questions). For data protection and equal opportunities, the estimated risk (five mentions each) overcomes the potential of improvement (three and four mentions, respectively); thus, these findings confirm that the potential discrimination of vulnerable users is a critical issue that needs to be addressed in the use of MATE systems, as described in the ELSI-R4 requirement. Data protection reported the smallest number of mentions for potential for improvement, which is clearly explained by the mission of a MATE system. Risks for occupational safety were mentioned by four users, and the smallest risks (three mentions) were reported for reintegration.

Moreover, each participant was asked how positively the impact of strain measurement was considered for each dimension of the ELSI concept. Results, measured on a scale from one (very negative) to ten (very positive), are reported in Figure 7. The questionnaire responses indicated that the impact of strain measurements according to each dimension was mostly rated positively (above six points), with the exception of two dimensions: data protection, ; and equal opportunities, . The influence of strain measurements on occupational health, , was rated highest. The impact of stress measurements on occupational safety was rated , and on ergonomic workplace design it was rated ; the impact on reintegration was rated .

Fig. 6: The potential improvement and potential risks reported for each dimension of the ELSI concept.
Fig. 7: The mean impact of strain measurement per dimension of the ELSI concept. The number of averaged answers is reported above each bar.

Iii Discussion and Conclusion

In this article, we propose a new human-centered approach based on affective robotics and computing, with advanced automatic or robotic system designs. We call the resulting system MATE. Measuring the operator’s skill, performance, and stamina, it then adapts itself to facilitate a specific interaction. Teaching and support are provided to the operator to train him or her on the skills necessary to implement the system effectively. This type of system is intended to be usable for all users —particularly the most vulnerable ones— thus satisfying the requirements of inclusive design.

The social and ethical implications related to the use of MATE systems are reported in this article, and we describe the requirements that are essential to design a MATE system. Technical requirements are summarized and discussed in tandem with the ethical, legal, and social recommendations and principles inspired by roboethics.

Several organizational factors that are affected by the use of a MATE system are identified along with their impact on a company’s structure. We then investigate how companies are currently organized in terms of such factors and what measures were previously taken to implement them.

Results indicated that more than 50% of the participating companies implement occupational health, safety, and data protection management systems; however, there is still a lack of implementation in ergonomic workplace design, equal opportunities, or reintegration, though the estimated relevance for all ELSI dimensions was rated positively. Conversely, Figure 5 also demonstrates that relevance was rated highest for those ELSI dimensions that are currently in practice and well known. Since the MATE approach attempts to support the elderly, the impaired, and the low-skilled worker during machine and robotics operations, it thereby has the potential to create changes in the awareness of equal opportunities and reintegration for companies in the future. This assumption is consistent with regard to the estimated impact, which was rated high for health, safety, ergonomics, and reintegration.

The impact on data protection was rated surprisingly low, as shown in Figure 7, although potential risks were expected to be greater than the potential for improvement, as shown in Figure 6. Equal opportunities play only a minor role compared to the other dimensions. The greatest potential for improvement is seen in health, safety, and ergonomics, which is consistent with the general objective of MATE systems. All of these dimensions were rated as having relatively low risks for the implementation of a MATE system.

With regard to the dimensions that are infrequently implemented, i.e., equal opportunities and reintegration, these were rated as having the lowest potential of improvement and were perceived to have the highest level of risk when compared to the potential for improvement. Therefore, a MATE system can not only demonstrate better awareness for ELSI factors, as mentioned previously, but it can also be mindful of specific dimensions that may have a greater level of risk and less potential for improvement when implemented in an industrial environment.

Lastly, the reliability of these results must be verified, and these issues should be examined further in subsequent investigations. Specifically, the proposed design recommendations and requirements will be reconsidered by assessing the real impact on concrete use cases brought by the MATE systems built in the framework of the INCLUSIVE project. The anticipated risks and improvements and possible side effects of strain measurement will be verified and updated, since possible other risks might arise, leading to further countermeasures to be considered in the design process.

Results show that more than 50% of the participating companies implement occupational health, safety and data protection management systems, but that there is still a lack of implementation in ergonomic workplace design, equal opportunities or reintegration, though estimated relevance for all ELSI dimensions was rated positively. On the other hand, Fig. 5 also shows that relevance was rated highest for those ELSI dimensions that are already implemented in practice and well known. Since the here discussed MATE approach aims at supporting elderly, impaired and unskilled workers during machine and robotics operations, it thereby has the potential to create changes in the awareness of equal opportunities and reintegration in the companies in the future. This assumption consolidates with regard to the estimated impact, which was rated high for health, safety, ergonomics and reintegration.

The impact on data protection was rated surprisingly low, as shown in Fig. 7, although potential risks were expected to be grater than improvement potential, as shown in Fig. 6. Also equal opportunities only plays a minor role compared to the other dimensions. Moreover, the highest potential of improvement is seen in health, safety and ergonomics, which is the general objective of MATE systems. All these dimensions were rated with relatively low risks for the implementation of a MATE system.

As regards the currently less implemented dimensions, namely equal opportunities and reintegration, they were rated with the lowest potential of improvement and with the highest risk if compared to potential improvement. Hence, a MATE system cannot only retrieve better awareness for ELSI questions, as concluded earlier, but also should consider to pay special attention towards dimensions that might have higher risk than potential of improvement, when implemented in industrial environment.

Finally, the reliability of these results has to be verified and these issues have to be inspected more in detail in further investigations. Specifically, the proposed design recommendations and requirements will be reconsidered by assessing the real impact on concrete use cases brought by the MATE systems built in the framework of the INCLUSIVE project. The foreseen risks and improvements and possible side effects of strain measurement will be verified and updated, since possible other risks might arise, thus leading to further counter measures to be considered in the design process.

Acknowledgement

The research is carried out within the “Smart and adaptive interfaces for INCLUSIVE work environment” project, funded by the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement N. 723373. The authors would like to express their gratitude for the support given.

References

  • [1] D. A. Norman, The design of everyday things: Revised and expanded edition.   Basic books, 2013.
  • [2] T. Skripcak, P. Tanuska, U. Konrad, and N. Schmeisser, “Toward nonconventional human-machine interfaces for supervisory plant process monitoring,” IEEE Trans. Human-Machine Systems, vol. 43, no. 5, pp. 437–450, Sep. 2013.
  • [3] F. Nachreiner, P. Nickel, and I. Meyer, “Human factors in process control systems: The design of human–machine interfaces,” Safety Science, vol. 44, pp. 5–26, 2006.
  • [4] G. Viano, A. Parodi, J. Alty, and et al., “Adaptive user interface for process control based on multi-agent approach,” in Proc. Working Conf. Advanced Visual Interfaces, ser. AVI ’00.   New York, NY, USA: ACM, 2000, pp. 201–204.
  • [5] R. W. Picard, Affective computing.   MIT Press Cambridge, 1997, vol. 252.
  • [6] C. Talignani Landi, V. Villani, F. Ferraguti, L. Sabattini, C. Secchi, and C. Fantuzzi, “Relieving operators’ workload: Towards affective robotics in industrial scenarios,” Mechatronics, vol. submitted, 2017.
  • [7] “The principles of universal design,” The Center for Universal Design, NC: North Carolina State University, Tech. Rep., 1997.
  • [8] C. Stephanidis and A. Savidis, “Universal access in the information society: Methods, tools, and interaction technologies,” Universal Access in the Information Society, vol. 1, no. 1, pp. 40–55, 2001.
  • [9] J. Abascal and C. Nicolle, “Moving towards inclusive design guidelines for socially and ethically aware HCI,” Intercating with Computers, vol. 17, pp. 484–505, 2005.
  • [10] V. Villani, L. Sabattini, J. N. Czerniak, A. Mertens, B. Vogel-Heuser, and C. Fantuzzi, “Towards modern inclusive factories: a methodology for the development of smart adaptive human-machine interfaces,” in 22nd IEEE Int. Conf. Emerging Technologies And Factory Automation (ETFA).   IEEE, 2017. [Online]. Available: https://arxiv.org/abs/1706.08467
  • [11] “Smart and adaptive interfaces for INCLUSIVE work environment.” [Online]. Available: http://www.inclusive-project.eu
  • [12] A. Sauppé and B. Mutlu, “The social impact of a robot co-worker in industrial settings,” in Proc. 33rd Annu. ACM Conf. Human Factors in Computing Systems (CHI).   ACM, 2015, pp. 3613–3622.
  • [13] S. A. Elprama, C. I. Jewell, A. Jacobs, I. El Makrini, and B. Vanderborght, “Attitudes of factory workers towards industrial and collaborative robots,” in Proc. ACM/IEEE Int. Conf. Human-Robot Interaction (HRI).   ACM, 2017, pp. 113–114.
  • [14] G. Veruggio, F. Operto, and G. Bekey, “Roboethics: Social and ethical implications,” in Springer handbook of robotics.   Springer, 2016, pp. 2135–2160.
  • [15] N. Nevejans, “European civil law rules in robotics,” Policy Department for “Citizens’ Rights and Constitutional Affairs”, European Union, Tech. Rep., Oct. 2016.
  • [16] L. Probst, L. Frideres, B. Pedersen, and C. Caputi, “Service innovation for smart industry - human-robot collaboration,” European Union, Business Innovation Observatory, Tech. Rep., Feb. 2015.
  • [17] A. Van Wynsberghe, “Service robots, care ethics, and design,” Ethics and Information Technology, vol. 18, no. 4, pp. 311–321, 2016.
  • [18] J. Leonard, “Systems engineering fundamentals,” Defense Systems Management Coll Fort Belvoir VA, Tech. Rep., 1999.
  • [19] L. Sabattini, V. Villani, J. N. Czerniak, A. Mertens, and C. Fantuzzi, “Methodological approach for the design of a complex inclusive human-machine system,” in 13th IEEE Conf. Automation Science and Engineering (CASE).   IEEE, 2017. [Online]. Available: https://arxiv.org/abs/1706.08461
  • [20] A. Manzeschke, K. Weber, E. Rother, and H. Fangerau, “Ethical questions in the area of age appropriate assisting systems,” German Federal Ministry of Education and Research, VDI/VDE Innovation+ Technik GmbH, 2015.