1.1 Social humans, social robots
Humans are inherently social beings, spending a great deal of their time establishing a diverse range of social connections. Their social nature is not only demonstrated by their social behavior homans1974social, but also possesses a biological basis frith2010social. This social dimension prompts human beings to involuntarily ascribe social qualities even to non-human media, such as technological artifacts, often treating them similarly to how they would treat humans or other living beings nass1994computers. This disposition stems from the general human tendency of ascribing human-like qualities to non-human entities, called anthropomorphism, which has been observed and demonstrated in several contexts epley2007seeing. These phenomena therefore place technologies capable of social interactions with humans as unique technological innovations. In particular, social robots, i.e., robots deliberately designed to interact with humans in a social way, open up a new paradigm for humans to communicate, interact, and relate to robotic technologies.
The integration of a social dimension in the design of robots has generally been following two approaches. First, existing robotic technologies are being enhanced with social capabilities for more fluid interactions with humans. Second, social robots are being developed for new application areas where the social dimension is central, and beyond a mere interface. As a result of these approaches, social robots have been deployed in a wide variety of contexts, such as healthcare broadbent2009acceptance, education belpaeme2018social, companionship dautenhahn2005robot, and others (refer to Section 2.3 for a discussion of application areas). They offer a spectrum of interactions that is being continuously enriched by researchers from a variety of disciplines. The field of human-robot interaction (HRI), as an expanding field of research, reflects this observation.
), machine learning,human-computer interaction (HCI), design, art, animation, cognitive science, psychology, sociology, ethology, and anthropology fong2002survey; murphy2010human; baxter2016characterising; alves2016psychological; eyssel2017experimental. The multidisciplinarity inherent to this field of research provides contributions and advancements nurtured by scholars from different backgrounds in the conception, design, and implementation of social robots. In addition to development, HRI aims to evaluate how well such robots perform or serve the purpose they were designed for, being concerned with proper evaluation, testing, and refinement of these technologies. The result is a rich multidisciplinary effort to create engaging robots that can sustain personalized interactions with humans, adapt to the task at hand and to the interaction flow, but also understand and model aspects pertaining to the humans, such as affect and cognition ho2010modelling; leite2013social.
In this chapter, we provide a framework for characterizing social robots that encompasses major aspects to consider when designing them and their interactions with humans. Our framework is focused on interactive robots that possess a social component in their design. Specifically, we use the term “social robots” to denote “socially interactive robots” as defined by Fong et al. fong2002survey, namely robots that have one or more of the following abilities: (1) communicating using natural language or non-verbal modalities (such as lights, movements, or sound), (2) expressing affective behaviors and/or perceiving human emotions, (3) possessing a distinctive personality or character, (4) modeling social aspects of humans, (5) learning and/or developing social skills, and (6) establishing and maintaining social relationships fong2002survey.
Our framework builds upon existing work within the field of HRI, providing a holistic understanding about the state of the art, while aiming at unifying, clarifying, and extending key concepts to be considered in the design of social robots. Specifically, our framework comprises several dimensions we identified to be of major relevance to the design of social robots. We summarize the dimensions considered in Figure 1. Some of these dimensions relate to the robot itself – namely appearance, social capabilities, and autonomy/intelligence –, others relate to the interaction – namely proximity and temporal profile –, and the remaining ones relate to the context – namely robot relational role and purpose / application area. We envision this framework to be used broadly in order to gain a better understanding of existing social robots, as well as to inform the design and development of future ones.
1.2 Brief summary of frameworks for characterizing social robots
Before outlining the content of our framework, it is useful to first look at existing frameworks for classifying social robots. In particular, existing taxonomies, as such from Fong et al.fong2002survey, Yanco et al. yanco2004classifying, Shibata shibata2004overview, and Dautenhahn dautenhahn2007socially, are useful to get an understanding of different aspects that may be included in the design space of social robots in HRI research. While this list of frameworks is not exhaustive, we chose these particular ones to base our framework on, as they provide a broad range of classifications and definitions that relates to the scope of this chapter.
As such, Fong et al. fong2002survey contributed a taxonomy of design methods and system components used to build socially interactive robots. These components include robot social capabilities, several design characteristics, and application domains.
Additionally, Yanco et al. yanco2004classifying provided a framework that included elements of social robot’s design, such as the role that a robot can have when interacting with humans, the types of tasks that robots can perform, different types of robot morphology, and the level of autonomy at which robots can operate.
Similarly, Shibata shibata2004overview provided a taxonomy for the function and purpose of social robots by considering different ways of using them for psychological enrichment. Therefore, Shibata classified human-robot interactions in terms of the duration of these interactions and in terms of design characteristics (e.g., robot’s appearance, hardware, and software functionalities), accounting for culture-sensitive aspects.
Moreover, Dautenhahn dautenhahn2007socially focused on different evaluation criteria to identify requirements on social skills for robots in different application domains. The author identified four criteria, including contact between the robot and the human (which can vary from no contact or remote contact to repeated long-term contact), the extent of the robot’s functionalities (which can vary from limited to a robot that learns and adapts), the role of the robot (which can vary between machine or tool to assistant, companion, or partner), and the requirement of social skills that a robot needs to have in a given application domain (which can vary from not required/desirable to essential). The author further explains that each evaluation criteria should be considered on a continuous scale.
Taken together, these classifications and taxonomies have gathered essential aspects for the characterization and design of social robots. Despite each of them being unique in its contribution, we can see the existence of some overlapping terms and ideas between them. We now discuss our extended framework in the next section.
1.3 Overview of our extended framework
Our framework leverages the existing ones discussed previously as a starting point and goes beyond the individual frameworks discussed. In particular, it focuses on the following features:
Unification — The existence of multiple available perspectives in HRI often results in scattered concepts and classifications. In this chapter, we aim at merging aspects of the literature on social robots and related fields in a self-contained and consistent resource.
Breadth — Existing individual taxonomies often focus on specific aspects relevant to the main line of research of their authors, and may not provide satisfactory coverage. Our framework includes dimensions related to the design of the robot itself, but also of the interaction and context.
Recency — In recent years, we have observed some important developments in robotic technologies, which have taken robots outside of research laboratory settings and enabled them to be deployed “in the wild”. We incorporate those recent developments in our work.
Clarity — Concepts associated with HRI are often difficult to define, and as a result clear definitions may not always be available. This lack of clarity may impede communication within the field, or result in inconsistent concepts. In this chapter, we attempt to clarify some important key concepts, such as the distinction between embodiment and purpose, or the concepts of autonomy and intelligence for social robots.
With these points in mind, we list below our focuses within each of the dimensions considered.
Appearance — We present a broad classification of robot appearances, synthesizing and going beyond existing ones (Section 2.1).
Social capabilities — We contribute a repositioning of existing classifications aiming to clarify how existing categories related to each other (Section 2.2).
Purpose and application area — We discuss a cross-section of purposes for social robots, and benefiting application areas, with selected examples that include recent developments in the field (Section 2.3).
Relational role — We provide a straightforward and broad classification of the robot’s role in relation to the human(s) (Section 2.4).
Autonomy and intelligence — We clarify the related but distinct concepts of autonomy and intelligence, and discuss their quantification (Section 2.5).
Proximity — We classify interactions according to their spatial features (Section 2.6).
Temporal profile — We look at several time-related aspects of the interaction, namely timespan, duration, and frequency (Section 2.7).
It is to be noted that our framework is not meant to be exhaustive, but rather to provide the reader with major aspects that shape social robots and their interactions with humans. While our focus in illustrating the presented concepts will be on single human - single robot interactions, the concepts may also apply for group interactions involving more than one robot and/or more than one human. Additionally, even though this framework was developed with social robots in mind, some dimensions may also be of relevance to robots without a social component in their design, such as for example in the “appearance” dimension. In the following section, we delve into each of the dimensions of our framework. We then end this chapter with a brief discussion on designing social robots within the resulting design space.
2 Framework description
We now provide a description of each of the dimensions of our framework. The dimensions purposefully operate at different levels, according to the aspects that are most relevant to the design of social robots. In some dimensions, we provide a classification into different categories and possibly subcategories (namely Sections 2.1, 2.3, 2.4, 2.6, and 2.7). In others, we focus on clarifying or reinterpreting existing distinctions in categories or scales (namely Sections 2.2 and 2.5). Due to different levels of research and relevant content in each, some dimensions are addressed in more depth than others. Also, since the discussions of dimensions are not dependent on each other, we invite the reader to jump to their subsections of interest.
The mere physical presence of robots in a shared time and space with humans sparks crucial aspects of a social interaction. Indeed, embodiment, a term used to refer to the idea that “intelligence cannot merely exist in the form of an abstract algorithm but requires a physical instantiation, a body” pfeifer2001understanding, plays an important role in the perception and experience of interacting with intelligent technology. Indeed, literature supports that physical embodiment influences the interaction between humans and robots lee2006physically; wainer2007embodiment; powers2007comparing; mumm2011designing; fasola2011comparing; fasola2011comparing; li2015benefit; kennedy2015comparing. In particular, the physical appearance of a robot per se, was shown to have a strong influence on people regarding aspects like perception, expectations, trust, engagement, motivation and usability jordan1998human; disalvo2003seduction; breazeal2004designing.
Several taxonomies were developed in order to create representative classifications for a robot’s appearance. To cite a few, Shibata shibata2004overview classified robots as being human type, familiar animal type, unfamiliar animal type, or imaginary animals / new character type. Additionally, Fong et al. fong2002survey considered anthropomorphic, zoomorphic, caricatured, and functional categories. The amount of classifications present in the literature urges for a unified and broad classification for social robot appearances. Building upon the existing classifications, we introduce a broad classification that encompasses main categories described by other authors, as well as new categories and subcategories. Our classification targets only and exclusively a robot’s physical appearance, as distinct from any type of robot behavior, i.e., “robot at rest”.
We contribute to the study of social robot’s appearance in the following ways: (1) we integrate similar terms already present in the robot appearance classification literature, (2) we add new terms to existing classifications as they were not represented in the literature but urged for a classification, and (3) we attempt to clarify concepts related to different categories. Our unified classification is visually represented in Figure 2. We considered the following categories of robot appearances: bio-inspired, including human-inspired and animal-inspired, artifact-shaped, and functional, each with several further subcategories (see Figure 2). We generated this classification with a holistic mindset, meaning it can serve to classify existing robots, but also to inform the design of future ones. Although devised with social robots in mind, it is general enough to be applied to any robot, independent of its social capabilities. We now provide a description of each category in our classification.
Bio-inspired — Robots in this category are designed after biological organisms or systems. This includes human-inspired and animal-inspired robots (described next), as well as other bio-inspired robots, such as robotic plants (e.g., the robotic flower111http://www.roboticgizmos.com/android-things-robotic-flower/) and fungi (e.g., the Lollipop Mushroom robot222https://www.amazon.com/Lollipop-Cleaner-Mushroom-Portable-Sweeper/dp/B01LXCBM3E).
Human-inspired — Robots in this category are inspired by features of the human body, including structure, shape, skin, and facial attributes. Human-inspired robots not only include full-body designs, but also robots designed after human body parts. When designed after the full-human body, they are called humanoids. The level of fidelity can vary from a highly mechanical appearance, such as the LOLA robot buschmann2009humanoid, to a highly human-like appearance that includes skins and clothes, such as the ERICA robot glas2016erica, or even include an intermediate between this two, in the case of the NAO robot333https://www.softbankrobotics.com/emea/en/nao. For humanoids, it is worth mentioning the case in which they strongly resemble the human outer appearance and are covered with flesh- or skin- like materials, in which case they are often referred to as androids (if they possess male physical features) or gynoids (if they possess female physical features). An example of a gynoid is the Kodomoroid robot444 http://www.geminoid.jp/en/robots.html. Additionally, a special case of androids/gynoids are geminoids, which are designed after an existing human individual – i.e., it is a “robotic twin” – such as Geminoid HI-4555http://www.geminoid.jp/projects/kibans/resources.html, the tele-operated robotic twin of Hiroshi Ishiguro. On the other hand, some robots are inspired by individual parts of the human body. These include robotic arms, e.g., Elumotion Humanoid Robotic Arm666http://elumotion.com/index.php/portfolio/project-title-1, robotic hands liu2007modular, robotic heads such as the EMYS robot kkedzierski2013emys, robotic torsos, shidujaman2018roboquin, and robotic facial features, such as robotic eyes cannata2006design,. It is worth mentioning that high-fidelity human-inspired robots are often subject to uncanny valley effects mori1970uncanny. Being highly but not totally human-like, they elicit feelings of eeriness, and hence should be designed bearing these possible effects in mind.
Animal-inspired — Robots in this category are inspired by animals or by creatures possessing animal traits of appearance. On the one hand, they may be inspired by real animals, for which we consider inspiration from familiar animals, like the AIBO777https://us.aibo.com/ dog-inspired robot, and inspiration from unfamiliar animals, such as the PARO888http://www.parorobots.com/ baby seal robot. The distinction between familiar and unfamiliar animals is emphasized by Shibata shibata2004overview. According to the author, familiar animals are those whose behavior can be easily recognized, such as pets; while unfamiliar animals are those that most people know something about but are not totally familiar with, and have rarely interacted with them before, such as savanna animals. The same author mentioned that when robots are designed to resemble an unfamiliar animal they can be more easily accepted due to the lack of exposure to their typical behavior. It is documented in the literature that people hold strong expectations when faced with the possibility of interacting with a social robot spence2014welcoming, wherein robots whose embodiment matches its abilities are perceived more positively goetz2003matching; li2010cross; komatsu2012does. However, it is to be noted that familiarity is a subjective concept depending on culture and individual experiences, making this distinction flexible. On the other hand, animal-inspired robots can also be imaginary, meaning they possess animal-like features but are not designed after a real animal. They can either be familiar, i.e., designed after familiar imaginary animals “existing” in fantasy worlds, like cartoon characters or legendary creatures (e.g., DragonBot short2014train), or unfamiliar, i.e., robots that are purely created from imagination, such as Miro999http://consequentialrobotics.com/miro/ and Keepon101010https://beatbots.net/my-keepon. In addition, this category includes robots designed after animal body parts, such as the GoQBot designed as a caterpillar part lin2011goqbot, the Meshworm designed after the oligochaeta seok2010peristaltic, and robotic soft tentacles jorgensen2018interaction.
Artifact-shaped — Robots in this category bear the appearance of physical human creations or inventions. They may be inspired by objects, such as furniture and everyday objects, e.g., the AUR robotic desk lamp hoffman2010effects, the Mechanical Ottoman robotic footstool sirkin2015mechanical, and the Travelmate robotic suitcase111111https://travelmaterobotics.com/. They may also be inspired by an existing apparatus, demonstrating how existing apparatuses can become robotic systems while maintaining the same appearance, such as self-driving cars (e.g., the Google self-driving car121212https://waymo.com/), but also everyday apparatuses like toasters, washing machine, etc. Additionally, artifact-shaped robot may be imaginary, i.e., translating the invention of the designer, such as the Greeting Machine robot anderson2018greeting or YOLO alves2017yolo; alves2018yolo.
Functional — The appearance of robots included in this category is merely the sum of appearances of the technological pieces needed to achieve a given task or function. This means that their appearance leans more towards mechanical aspects. Examples are quadcopters, or mobile robots such as the CoBots veloso2015cobots, the TurtleBot131313https://www.turtlebot.com/, and the Beam141414https://suitabletech.com/.
As a side note, shape-shifting robots, modular robots, or polymorphic robots balch2002robot; yim2002modular; yim2007modular; li2009amoeba are all examples of hybrid robots that can fit into more than one category depending on their configuration. Also, robotic swarms are examples of multi-robot systems that may be perceived as a single entity, i.e., more than the sum of individual robots (homogeneous or heterogeneous) kolling2016human, however they are they are not part of our classification, because they are too dependent on the configuration and behavior of the swarm. Moreover, the actual process of assigning categories to existing robots always carries a certain degree of subjectivity, which relates to different possible perceptions of the same robot appearance, depending or not on the context, the behavior of the robot, and so on. The clearest example in our classification would be the distinction between familiar and unfamiliar, which strongly depends on people’s cultural background and personal experiences. Those differences in perception should be accounted for when designing robot appearances.
Our presented classification is not intended to offer a clear-cut or rigid boundary between categories of robots. Rather, it represents a useful guideline for categorizing robots based on major distinguishing features. It does encourage the view of robot design as a spectrum, providing fluidity to their design and allowing for the combination of different elements of our classification.
A robot’s appearance is the most obvious and unique visual attribute, which contributes highly to the interaction fink2012anthropomorphism. Nonetheless, in addition to appearance, there are several factors related to embodiment, such as size, weight, noise, material texture, among others disalvo2002all that may contribute to the perception of the robot during an interaction. More research is needed in order to develop classifications that account for the other factors mentioned above.
2.2 Social capabilities
Social robots vary greatly in their social capabilities, i.e., how they can engage in and maintain social interactions of varying complexities. As such, researchers have classified and defined them according to those social capabilities. Based on the work of Fong et al. fong2002survey, we list the different components of a social robot’s capabilities as follows:
Communicating using natural language or non-verbal modalities — Examples of these ways of communication are natural speech williams2018thank, motion knight2011eight; dragan2013legibility – possibly including gaze admoni2017social, gestures or facial expressions –, lights baraka2018mobile; szafir2015communicating, sounds bethel2006auditory, or a combination of them loffler2018multimodal. Mavridis mavridis2015review provided a review on verbal and non-verbal interactive communication between humans and robots, defining different types of existing communications such as interaction grounding, affective communications, speech for purpose and planning, among others.
Expressing affect and/or perceiving human emotions — Beyond Ekamn’s five basic emotions ekman1992argument – anger, disgust, fear, happiness, sadness, and surprise –, this may include more complex affective responses such as empathy. For example, Paiva et al. paiva2017empathy analyzed different ways by which robots and other artificial agents can simulate and trigger empathy in their interactions with humans.
Exhibiting distinctive personality and character traits — The major components to be considered, according to Robert robert2018personality, are human personality when interacting with a robot, robot personality when interacting with humans, dissimilarities or complementary in human-robot personalities, and aspects that facilitate robot personality. Some companies such as Misty Robotics 151515https://www.mistyrobotics.com/ are prioritizing the user personalization of a robot’s personality as an important feature for future commercial social robots.
Modeling and recognizing social aspects of humans — Modeling human agents allows for robots to interpret aspects of human behavior or communication and appropriately respond to them. Rossi et al. rossi2017user provide a survey of sample works aimed at profiling users according to different types of features. More advanced models may have to consider theory of mind approaches scassellati2002theory.
Learning and developing new social skills and competencies — In addition to being programmed to have social skills, social robots may have the ability to refine those skills with time through adaptation, or even developing new skills altogether. An active area of research that looks at such paradigms is the area of developmental robotics lungarella2003beyond.
Establishing and maintaining social relationships — Relationships operate over a timespan that goes beyond a few interactions. A number of questions arise when one considers long-term interactions of robots with humans and what it means for a robot to proactively establish and maintain a relationship that is two-sided. Leite et al. leite2013social established some initial guidelines for the design of social robots for long-term interaction. These include continuity and incremental robot behaviors (e.g., recalling previous activities and self-disclosure), affective interactions and empathy (e.g., displaying contextualized affective reactions), and memory and adaptation (e.g., identifying new and repeated users).
Complementary to these components, Breazeal breazeal2003toward distinguished categories of robot social capabilities: (1) socially evocative, denoting robots that were designed mainly to evoke social and emotional responses in humans, leveraging the human tendency to anthropomorphize epley2007seeing. Therefore, despite expected social responsiveness, the robot’s behavior does not necessarily reciprocate; (2) social interface, denoting robots that provide a “natural” interface by using human-like social cues and communication modalities. In this sense, the social behavior of humans is only modeled at the interface level, which normally results in shallow models of social cognition in the robot; (3) socially receptive, denoting robots that are socially passive but that can benefit from interaction. This category of robots is more aware of human behavior, allowing humans to shape the behavior of the robot using different modalities, such as learning by demonstration. Also, these robots are socially passive, responding to humans’ efforts without being socially pro-active; and (4) sociable, denoting robots that pro-actively engage with humans, having their own internal goals and needs in order to satisfy internal social aims (drives, emotions, etc.). These robots require deep models of social cognition not only in terms of perception but also of human modelling.
In addition to this list, Fong et al. fong2002survey added the following categories: (5) socially situated, denoting robots that are surrounded by a social environment that they can perceive and react to. These robots must be able to distinguish between other social agents and different objects that exist in the environment; (6) socially embedded, denoting robots that are situated in a social environment and interact with other artificial agents and humans. Additionally, these robots can be structurally coupled with their social environment, and have partial awareness of human interactional structures, such as the ability to perform turn-taking; and (7) socially intelligent, including robots that present aspects of human-style social intelligence, which is based on deep models of human cognition and social competence.
Although robots have been classified according to their different social capabilities, it is yet unclear how these categories relate to each other. Are they part of a spectrum? Are they separate categories altogether? We argue that evaluating social capabilities of robots can be understood according to two main dimensions:
The depth of the robot’s actual social cognition mechanisms.
The human perception of the robot’s social aptitude.
Given these dimensions, and in light of the existing categories presented above, we propose a two-dimensional space map, providing a clearer understanding of the social capabilities of robots. This map is presented in Figure 3 for illustrative purposes. As it can be seen in the figure, socially evocative robots have the least depth of social cognition but are perceived as rather socially apt. A social interface typically possesses some additional cognition mechanisms to allow for easy communication with the range of the robot’s functionality; it also possibly results in a slightly higher perceived social aptitude thanks to its more versatile nature. Socially receptive robots, socially situated, and socially embedded robots possess increasing depth in their social cognition, and as a result increasing perceived social aptitude. For socially embedded robots, the perceived aptitude may vary according to the degree of awareness about interactional structure the robot has. On the outskirts of our map we find sociable and socially intelligent robots, with much deeper models of social cognition.
2.3 Purpose and application area
In this section, we discuss social robots according to their purpose, i.e., what types of goals they are designed to achieve, as well as benefiting application areas. Figure 16 summarizes the main purposes and application areas included in this section, with illustrative examples.
A note on purpose as being distinct from embodiment
In traditional engineering practice, the physical characteristics of a technological device (e.g., toaster, microwave, typewriter, manufacturing machine) tend to be strongly coupled with its purpose, i.e., the task it was designed to achieve. With the advent of personal computers and smartphones, we moved away from defining those devices solely by their purpose. For instance, it would be inappropriate to call a modern computer an “electronic typewriter” or even a smartphone an “electronic phone”, because those devices can serve an immense variety of uses, thanks to software applications that constantly create new purposes for them. Similarly, even though some robots may currently be designed for a specific purpose in mind, some robots may possess a set of skills that can prove useful in a variety of scenarios, sometimes across completely different application areas. As a result, (1) many different robots can be programmed to be used for the same purpose, but also (2) a single robot can be used for many different purposes. For example, a robot such as NAO has been used across a large variety of purposes, both in research and industry, from playing soccer graf2009robust to assisting individuals with cognitive impairments shamsuddin2012initial; esteban2017build or teaching children yadollahi2018deictic; alves2019empathic.
There remains, however, a general tendency to define robots by characteristics of their programmed behavior, which can be limiting or inappropriate. As an example, we see locutions of the form “educational robots”, “therapeutic robots”, “pet robots”, and so on. The Baxter robot171717https://www.rethinkrobotics.com/baxter/, for instance, is often referred to as a “collaborative industrial robot” (or co-bot), because it has been used quite often in such a setting. However, it has also been used in very different applications, such as assistance for the blind bonani2018my, or education fernandez2018may, and hence the naming is reductive. Similarly, a “pet robot” such as the AIBO dog-inspired robot has been used in contexts where it is far from being considered a pet, such as playing soccer with other robots stone2007intelligent.
Of course, the embodiment of the robot may restrict its capabilities and hence the type of tasks it may be able to physically achieve. Also, the robot’s hardware may be optimized for a specific interactive application (e.g., Baxter has compliant joints for safer collaboration). Moreover, a robot’s appearance, which goes beyond its hardware specifications, may be optimized for human perceptions such as acceptability, likeability, trust, and so on, for a specific intended purpose. However, given the considerations above, we believe that robots should not be defined solely by their purpose, the same way humans are (hopefully) not defined by their profession. As a result, we personally prefer a slightly different language to characterize robots according to their purpose(s): “robots for education” instead of “educational robots”, “robots for therapy” instead of “therapeutic robots”, and so on. Using this slightly modified language, we now discuss the main purposes and application areas that are benefiting from the use of social robots. In light of our discussion, the presented list is not meant to be selective, as the same robot may be used for more than one purpose.
Robots for healthcare and therapy
Robots are being introduced in the health sector to assist patients and providers in hospitals, at home, or in therapy settings. The type of assistance the robot provides can be generally categorized into physical and/or social. Physically assistive applications include helping patients with reduced mobility or dexterity, such as the elderly forlizzi2004assistive or people with physical impairments burgar2000development. These robots can help to carry out daily tasks, like getting out of bed, manipulating objects, eating, and so on, which can give them a higher sense of autonomy and dignity sharkey2012granny. They may also help in therapy to assist patients in regaining lost physical skills or build new ones burgar2000development. On the other hand, socially assistive robotics (SAR) focus on providing assistance primarily through social interactions. Feil-Seifer et al. feil2005defining identified a number of applications where SAR may have a strong impact, namely in therapy for individuals with cognitive disorders scassellati2012robots; cabibihan2013robots, companionship to the elderly and individuals with neurological disorders or in convalescent care burton2013dolphins, and students in special education. We also believe that robots in the healthcare domain may be used to benefit healthcare providers directly, for example training therapists through robotic simulation of interactions with patients baraka2019interactive.
Robots for education
Robots in education are mainly used with children kanda2007two; tanaka2007socialization; Kozima08aplayful because they can increase engagement in learning while favoring an interactive and playful component, which may be lacking in a traditional classroom setting. When designing such educational robots, it is crucial to design for and evaluate long-term interactions, to avoid successes merely due to strong novelty effects leite2013social.
There is a number of formats that educational scenarios can take, where the robot has a different role. Beyond being a teacher delivering material, the robot can also act as a social mediator between children, encouraging dyadic, triadic, and group interactions kozima2009keepon. Moreover, the robot may play the role of a learner in learning-by-teaching scenarios, in which the child teaches the robot and in this process develops their own skills jacq2016building.
Robots for entertainment and the arts
The entertainment industry has benefited from the use of robots for their engaging and interactive capabilities. Personal entertainment creations emerged with robotic toys, such as Furby181818https://furby.hasbro.com/en-us or Bee-Bot191919https://www.bee-bot.us/, and robotic dolls, such as Hasbro’s My Real Baby202020https://babyalive.hasbro.com/. Public entertainment robots have appeared in theme parks and other public entertainment spaces madhani2009bringing. More complex robots with both verbal and non-verbal communication capabilities have been used for more prolonged interaction scenarios such as storytelling chen2011survey or comedy bruce2000robot. Other entertainment applications include interactive shows alonso2014human, acrobatic robots for movie stunts pope2018stickman, and sex robots levy2009love, among others.
More artistic-oriented applications include robots in the visual arts212121An annual robot art competition is held to encourage the use of robots in the visual arts http://robotart.org/ pagliarini2009development and installation art augugliaro2014flight. Social robots have also been deployed in fields of performative arts such as drama zeglin2014herb or dance sum2017robot; cappo2018online, where their embodied intelligence in real-time contexts and their interactivity remain a challenging and rich research challenge. Generally, the inclusion of intelligent robots in the arts and the broader field of computational creativity colton2012computational are questioning definitions and criteria of art, authorship, and creativity.
Robots for industry
As industrial robots are becoming more intelligent, they are being equipped with interactional capabilities that allow them to collaborate with humans, mainly in tasks involving manipulation skills. Schou et al. schou2018skill identified several tasks that can benefit from a human-robot collaborative setting, possibly including multi-robot/multi-human teams. These are: logistic tasks (namely transportation and part feeding), assistive tasks (namely machine tending, (pre)assembly, inspection, and process execution), and service tasks (namely maintenance and cleaning).
Research has shown that robots exhibiting social communication cues in industrial settings are perceived as social entities sauppe2015social. Moreover, Fong et al. fong2002survey emphasized that, in order to achieve true collaboration between humans and robots, the robot must have sufficient introspection to detect its own limitations, must enable bidirectional communication and information exchange, and must be able to adapt to a variety of humans from the novice to the experienced.
Robots for search and rescue
Search and rescue is one of the applications in which robots are being investigated as replacements to humans in dangerous environments, such as in natural or human disasters. Even though typical robots in this domain have not been designed with social capabilities, research has shown the importance of “social intelligence” in this domain fincannon2004evidence. Bethel et al. bethel2008survey identified the importance of different modalities of social communication in the context of victim approach, across the scale of proxemic zones (i.e., the distancing of the robot to the human), ranging from the public to the personal space. Such modalities include body movement, posture, orientation, color, and sound.
Robots for assistance in home and workplace
With the advent of personal robots gates2007robot, the vision is that anyone will have the ability to own and operate a robot, regardless of their skills or experience, thanks to natural and intuitive interfaces liang2018simultaneous. Such robots can be deployed in home or workplace environments to assist individuals, reduce their mental and physical load, and increase their comfort and productivity. In the home, personal robots are already cleaning floor surfaces autonomously222222https://www.irobot.com/for-the-home/vacuuming/roomba, cooking full meals232323http://www.moley.com/, and doing laundry242424http://www.laundry-robotics.com/, just to name a few. More ambitious research projects have aimed at designing versatile “robotic butlers” srinivasa2010herb, that can operate in a variety of tasks across the home.
In the workplace, robots are being used on a daily basis to transport objects, cataloguing inventory, escorting people, delivering messages, among other tasks, in settings such as offices, hospitals252525https://aethon.com/, supermarkets262626http://www.bossanova.com, and hotels. The majority of these robots are called service robots and have the capability of navigating in structured indoor environments, mainly corridors as opposed to open public spaces. An example of such service robots is the CoBots veloso2015cobots, developed and deployed at Carnegie Mellon University, servicing multiple floors and having navigated more than km autonomously biswas20161. Other types of robots used in the workplace include tele-presence robots for teleconferencing and virtual visits of remote places tsui2011exploring.
Robots for public service
Robots have been deployed in public spaces including malls shiomi2009field, museums faber2009humanoid, exhibition spaces jensen2005robots, and receptions gockley2005designing. Some (but not all) of those robots are mobile, and can navigate in open spaces or in crowds, which makes the design of their behavior challenging and subject to a variety of social constraints luber2012socially. Interactions with such robots have to account for the fact that the robot will interact with a very large number of people, with inevitable differences, and during a short duration. Hence, personalizing the interaction and making it as intuitive as possible (as there is very little adaptation time on the human side) are important design considerations.
Robots for the social sciences
Due to the possibility of programming robots to exhibit mechanisms of cognition similar to those of humans, a less publicized purpose of robots is in fields of the social sciences for the study of social development, social interaction, emotion, attachment, and personality fong2002survey. The idea is to use robots as test subjects in controlled laboratory experiments, leveraging the fact that such robots can reproduce consistent behaviors repeatedly and can be controlled to test predictions of human models of cognition. For example, the Cog robot scassellati2003investigating was used to investigate models of human social cognition. Similarly, a doll-like robot, Robota billard2007building, was used in comparative studies for social development theories dautenhahn1999studying. Additionally, robots (human-inspired or other types) can be used as stimuli to elicit behaviors from humans for the development and refinement of theories about human behavior and cognition. For a more detailed discussion on cognitive robotics and its applications outside of technology-related fields, consult Lungarella et al. lungarella2003developmental.
Other application areas
The list of application areas and purposes listed above is not comprehensive, but reflects major developments and deployments. To this list we can add:
Robots for companionship — Dautenhahn dautenhahn2004robots presented a perspective on different possible relationships with personalized (possibly life-long) robotic companions, drawing on literature from human-animal relationships. Situated somewhere between animal pets and lifeless stuffed animals, robotic companions may provide support for socially isolated populations. The technical and design challenges associated with robotic companions are numerous due to the time dimension, and the deployment of robotic pets has raised an ethical concern sparrow2002march. Examples of robotic companions include the Huggable robot stiehl2009huggable, the AIBO dog-inspired robot friedman2003hardware, and the Lovot robot272727https://groove-x.com/en/.
Robots for personal empowerment — The ultimate ethically concerned use of robots is to expand human abilities instead of replacing them, and to empower people at an individual level. Examples of personal empowerment that robots may facilitate are physically assistive robots that help people with impairments gain autonomy and dignity, such as prosthetics, exoskeletons, brain-controlled robotic arms hochberg2012reach, and other assistive robots (see Section 2.3). Other examples include robots that are designed to enhance creativity in individuals, such as the YOLO robot alves2018yolo, or tele-presence robots for workers that cannot physically perform the required tasks, such as in the “Dawn ver. ” cafe in Japan who hired paralyzed people to serve the costumers through a mobile robot controlled by their eye movements282828https://www.bbc.com/news/technology-46466531.
Robots for transportation — The rise of autonomous driving will revolutionize transportation and the urban environment. Autonomous vehicles (cars, trucks, public transportation, etc.) are expected to operate in environments populated by humans (drivers, pedestrians, bicyclists, etc.), and research is looking at adding social dimensions to their behavior nass2005improving; wei2013autonomous; mavrogiannis2019effects. Additionally, drones will be used in the near future for package delivery292929https://www.amazon.com/Amazon-Prime-Air/b?ie=UTF8&node=8037720011 and will have to (socially) interact with costumers.
Robots for space — Robots for space exploration are historically known for their low level of interactions with humans. However, as humans are getting more involved in space explorations, social robots are being introduced to assist astronauts in their tasks and daily routines, e.g., the Jet Propulsion Laboratory’s Robonaut and Valkyrie yamokoski2019robonaut.
Robots for technology research — Robots can also be used to test theories in fields related to technology, such as testing algorithms and architectures on physical platforms. More generally, robots can provide a platform for developing and testing new ideas, theories, solutions, prototypes, etc., for effective embodied technological solutions and their adoption in society.
The application areas mentioned above provide a cross-section of purposes that social robots hold in existing developments and deployments. If we view robots as embodied agents that can carry intelligently complex tasks in the physical and social world, we expect, in the future, to have robots introduced in virtually any application where they can complement, assist, and collaborate with humans in existing roles and expand their capabilities, as well as potentially assume new roles that humans cannot or should not assume.
2.4 Relational role
One of the relevant dimensions that shapes human-robot interaction is the role that the robot is designed to fulfill. The concept of role is an abstract one, for which various different perspectives can be presented. In this section, we specifically look at the relational role of the robot towards the human. This is the role that a robot is designed to fulfill within an interaction, and is not necessarily tied to an application area. The relational role the robot has been designed to have is critical to the perception, or even the relationship, that arises between robot and human.
Towards clarifying the concept of relational role, it is important to immediately distinguish relational role from role in an activity or application. In a specific activity or application, we may expect to find activity-specific roles (as in role-playing), such as teacher, driver, game companion, cook, or therapist. These types of roles are defined by the type of activity performed between the robot and humans, therefore making it an open-ended list that is likely to stay in constant evolution, as robots become applied to new fields and tasks.
Given the fuzziness of this concept, there have not been many attempts at generalizing the concept of role of robots within a relation with humans. For the rest of this section, we will present and analyze some broader definitions from the existing literature, to conclude by contributing a broad classification that attempts to agglomerate the main concepts of the pre-existing ones while containing and extending them.
Scholtz et al. presented a list of interaction models found in HRI scholtz2003theory. They included roles that humans may have towards a robot in any HRI application. The list defines the roles of the Supervisor, who monitors and controls the overall system (single or multiple robots), while acting upon the system’s goals/intentions; the Operator, who controls the task indirectly, by triggering actions (from a set of pre-approved ones), while determining if these actions are being carried out correctly by the robot(s); the Mechanic, who is called upon to control the task, robot and environment directly, by performing changes to the actual hardware of physical set-up; the Peer, who takes part in the task or interaction, while suggesting goals/intentions for the supervisor to perform; and the Bystander, who may take part in the task or interaction through a subset of the available actions, while most likely not previously informed about which those are. These five roles were initially adapted from HCI research, namely from Norman’s HCI Model norman1986cognitive. As such, they refer mostly to the role of the human within a technological system, whereas in this section we look for a classification to support the roles of robots in relation to humans within their interaction with each other.
Later, Goodrich et al. goodrich2008human built upon this list to propose a classification of roles that robots can assume in HRI. In their list, it is not specified whether the role refers to a human or to a robot. Their proposed classification can be vague, as they take Scholtz’s roles (for humans) and directly apply them to both robots and humans with no discussion provided. They also extended the list by adding two more roles, but these are defined only for robot. In the Mentor role, the robot is in a teaching or leadership role for the human; in the Informer role, the robot is not controlled by the human, but the human uses information coming from the robot, for example in a reconnaissance task.
The concept of robot roles was also addressed by Breazeal breazeal2004social, who proposed four interaction paradigms of HRI. In these paradigms, the robot can either take the role of a Tool, directed at performing specific tasks, with various levels of autonomy; a Cyborg extension, in which it is physically merged with the human in a way that the person accepts it as an integral part of their body; an Avatar, through which the person can project themselves in order to communicate with another from far away; or a Sociable partner, as in classic science-fiction fantasy.
Based on the many different proposed classifications, and of all the various interaction scenarios and applications found throughout literature and presented throughout this chapter, we have outlined our own classification for the role of robots within a relation with humans. Our classification attempts to merge the various dimensions of interaction while stepping away from explicit types of scenarios or applications. It does not necessarily add or propose new roles, but instead, redefines them from a relational perspective, placing emphasis on how the robot relates from a human’s perspective, as depicted in Figure 5.
In our classification for relational roles of robots, we view HRI as including both robot and you (the human). As such, we consider the following roles that a robot may have towards you:
A robot “for you” serves some utility on a given task. This is the most traditional role of a tool or a servant, and is inspired by most previous classifications. Despite closely related with the concept of a tool, as proposed by other authors, we frame this role as a broader type of robotic tool, which can even include robots like autonomous cars.
A robot “as you” plays the role of a proxy, namely, but not limited to, tele-presence. However it does not necessarily imply interaction from far away as in Breazeal’s classification breazeal2004social. This type of role can exist even when inter-actors are co-located, as long as the robot is acting in place of another person who operates it (e.g. shared autonomy scenarios).
A robot “with you” is typically collaborative, with various levels of autonomy, including being part of a group with you. It is used in applications in which both the human and the robot act together, as a team, or towards common goals, and also includes robots for companionship. The robot and human are not necessarily co-located, such as for example human-robot teams that have to communicate remotely.
A robot “as if you” emulates particular social or psychological traits found in humans. These robots are mainly used as social sciences research tools (see Section 2.3.8). To date, robots have been used to examine, validate and refine theories of social and biological development, psychology, neurobiology, emotional and non-verbal communication, and social interaction.
A robot “around you”, shares a physical space and common resources with the human. It differs from a robot with you by the fact that it is necessarily co-located with the human, but not necessarily collaborating with them. These are typically called co-operating, co-present, or bystanders, as previously proposed in Scholzt’s classification scholtz2003theory.
A robot “as part of you” extends the human body’s capabilities. These robots typically have nonexistent or very limited autonomy, but provide humans with physical capabilities that they could not otherwise perform using their own biological body. Such robots can be used for pure embodiment extension (e.g. strength-enhancing exoskeletons), or for close-range HRI collaboration, such as the robotic wearable forearm vatsal2018design whose function is to serve as a supernumerary third arm for shared workspace activities.
The list of relational roles that we present defines non-exclusive roles, meaning that for some particular applications, we may design and develop robots that take more than one of these roles, or take a different role when more than one human is involved in the interaction. An example would be of a robot used in an office, which can be used for the users to deliver mail and packages to different locations, while at the same time acting around the users when navigating the office space. Another example would be an autonomous vehicle operating for the passenger(s), but around pedestrians and other human drivers.
2.5 Autonomy and intelligence
Necessary aspects to consider when characterizing the behavior of social robots are those of autonomy and intelligence. Although related, these are two distinct concepts that are often inconsistently and confusingly used in existing literature gunderson2004intelligence; gardner1996intelligence. In particular, it is often assumed that a high level of robot autonomy implies both a high level of intelligence and of complexity. In reality, some fully autonomous systems can possess very low intelligence (e.g., a traditional manufacturing machine) or complexity (e.g., a simple self-operated mechanism). A better clarification of the concepts of autonomy and intelligence, and their relation, is needed, especially in the context of social robotics.
Definitions (or lack thereof)
The concepts of autonomy and intelligence are hard to define, and there does not seem to be unique accepted definitions beer2014toward. In particular, existing definitions in the literature seem to differ depending on the context of application, and the main field of focus of the author(s). Based on existing literature, we propose below extended working definitions of those two concepts in the context of social robotics.
It may seem somewhat paradoxical to talk about autonomy in the context of interactive robots, because traditionally fully autonomous robots are involved in minimal interactions with humans; in other words, reduced interaction with humans is a by-product of increased robot autonomy. For social robots however, this relation between amount of human interaction and robot autonomy is questioned. Highly autonomous social robots are expected to carry out more fluid, natural, and complex interactions, which does not make them any less autonomous. There exists a very large number of definitions of autonomy for general agents, however central to most existing definitions is the amount of control the robot has over performing the task(s) it was designed to fulfill (or that it sets to itself), as emphasized by Beer et al. beer2014toward. For social robots, tasks may include well-defined goal states (e.g., assembling furniture) or more elusive ones (e.g., engaging in conversation).
We claim that in addition to control, the concept of autonomy should also account for learning. Indeed, many learning paradigms include human-in-the-loop approaches, and we believe these should taken into account. These include active learningchao2010transparent, learning by demonstration rybski2007interactive, and corrective human feedback learning mericcli2011task, used within the context of interactions in applications involving human teachers such as learning-by-teaching educational scenarios jacq2016building or general collaborative scenarios breazeal2004teaching. As a result, we extend the definition from Beer et al. beer2014toward to make it applicable to social robots, and define autonomy of a social robot as follows:
Autonomy — “The extent to which a robot can operate in the tasks it was designed for (or that it creates for itself) without external intervention.”
Note the use of the term intervention as opposed to interaction.
The is no real consensus on the definition of general intelligence gardner1996intelligence. In the context of robotics and AI, intelligence is generally emphasized as related to problem solving newell1972human. For social robots, we propose the following extension of the definition of Gunderson et al. gunderson2004intelligence:
Intelligence — “The ability to determine behavior that will maximize the likelihood of goal satisfaction under dynamic and uncertain conditions, linked to the environment and the interaction with other (possibly human) agents.”
Note that intelligence is also dependent on the difficulty of the goals to be achieved. Based on this definition, it can be seen that intelligence and autonomy are distinct concepts, but that, for a given task, intelligence creates a bound on achievable autonomy. In other words, the level of intelligence of a robot may prevent its ability to reach a given level of autonomy for fixed robot capabilities gunderson2004intelligence. A final important note concerning the design of social robots is that a robot’s perceived intelligence bartneck2009measurement can be drastically different from its actual intelligence. As a result, minimizing the gap between the two is crucial for maintaining adequate expectations and appropriate levels of trust on the human side. Now that we have defined the concepts of autonomy and intelligence, we discuss approaches to quantify them.
Quantifying autonomy and intelligence
Unlike scales from the automation endsley1999level or tele-operation sheridan1978human; huang2005autonomy; yanco2004classifying; goodrich2003seven fields, and more recently with autonomous vehicles sae2014automated, all of which are based on the idea that more autonomy requires less HRI, some researchers have developed scales of autonomy that apply to social robots beer2014toward; thrun2004toward; feil2007benchmarks; goodrich2008human. These emphasize on the fact that autonomy has to be understood as a dynamic entity goodrich2008human. On the other hand, measuring robot intelligence has been the subject of some investigation, from both practical adams2016athlon and theoretical perspectives bien2002machine. Both autonomy and intelligence can be seen as belonging to a continuum, taking into account aspects of robot perception, cognition, execution, and learning gunderson2004intelligence; yanco2004classifying. As a result, autonomy is a dimension that one designs for, constrained by possible achievable levels of intelligence. As a general rule, the higher the autonomy and intelligence is, the higher the complexity of the system is.
The importance of dimensional thinking
For a highly heterogeneous technology such as a social robot that involves a combination of hardware, software architecture, cognition mechanisms, intelligent hardware control, just to name a few, it is important to define dimensions about aspects such as autonomy and intelligence. The overall assessment of these aspects would then depend on a combination of assessments over individual dimensions. Researchers at IBM have proposed to define “dimensions of (general artificial) intelligence”, as a way to define an updated version of the Turing test turing2009computing. Their list is more task-oriented, but can serve as a basis to think about general dimensions for both intelligence and autonomy. We propose the following dimensions of intelligence and autonomy, accounting for the socially interactive factor:
Perception of environment-related and human-related factors —
In order to engage in successful interactions, social robots need to be able to assess the dynamic state of the physical environment and of humans, to inform its decision making. On the human side, this includes estimating the human’s physical parameters (pose, speed, motion, etc.), speech, and non-verbal social cues (gestures, gaze, prosody, facial expressions, etc.).
Modeling of environment and human(s) — In order to interpret robot perceptions, models of the environment and of humans are needed. For example, models of the humans can allow the robot to infer their intents, personality, emotional or affective states, and predict future human states or behavior. If models are parametrized to capture individual differences, then they can be a powerful tool to inform personalization and adaptation mechanisms in HRI rossi2017user.
Planning actions to interact with environment and human(s) — Decision-making on a robot can be reduced to creating plans for robot actions that take into account the shape of the task, the goal, and the current state of the world, including the robot, the environment, and the human(s). A social robot needs to plan its motion, speech, and any other modality of social behavior it may be able to exhibit.
Executing plans under physical and social constraints — The same way the environment poses physical constraints on how the robot interacts with it, culture and society impose social constraints on how interactions with a robot should take place lee2014culturally. Robot decision-making should take human social norms into account while planning and executing generated plans carlucci2015explicit. Note that the execution of the plan may not be successful, hence the robot needs to account for all possible outcomes.
Learning through interaction with the environment or humans — On top of the basic dimensions mentioned above, some robots may be endowed with learning capabilities, which allow them to improve with time, throughout their interactions with the environment or humans (including human-in-the-loop learning). Note that this dimension does not necessarily encompass machine learning as a general technique, as many offline machine learning methods would fall under the dimensions of perception and modeling.
The dimensions above span most existing building blocks for the intelligence of a social robot. However, depending on their implementation and complexity, some robots may not include one or more of the above dimensions. Those dimensions are generally separated in the design and implementation of most robots, hence as a result, intelligence and autonomy on each dimension may be completely different. For example, some semi-autonomous robots include completely human-controlled perception steinfeld2009oz, or rely on human input for learning chao2010transparent; rybski2007interactive; mericcli2011task or verifying the suitability of robot plans esteban2017build.
As technology advances, higher amounts of robot intelligence will be achievable, unlocking new possible levels of autonomy for more complex tasks; however, the amount of autonomy of a system (within possible technological limits) will remain a design choice. As a design principle for future social robots, we advocate for the notion of symbiotic autonomy veloso2015cobots; coradeschi2006symbiotic, where both humans and robots can overcome their limitations and potentially learn from each other.
Spatial features of the interaction may have a strong influence on the type of possible interactions and their perception by humans. In this section, we focus on the proximity of the interaction, i.e., the physical distance between the robot and the human. In particular, we consider general categories of interactions according to the proximity dimension: remote, co-located, and physical.
Several applications in HRI require the human and the robot to be in physically remote places. Tele-operation applications generally involve tasks or environments that are dangerous or inaccessible for humans, and historically represents one of the first involvements of humans with robots. In traditional tele-operation contexts, the human is treated as an operator, intervening to shape the behavior of one or more robots. Such types of HRI scenarios have been extensively studied and a number of metrics have been developed for them steinfeld2006common. However, they are often excluded from the literature in social robotics fong2002survey.
More recent developments in the field of tele-operation gave rise to tele-presence applications, which treat the robot as a physical proxy for the human tsui2011exploring; kristoffersson2013review, allowing the latter for example to be virtually present in tele-conferencing settings, or to visit remote places. As a result, as the robot is used to interact with humans in the remote environment, its design may include a strong focus on socially embodied aspects of the interaction beyond mere audio and video, such as distancing and gaze behavior adalgeirsson2010mebot.
In all the previously cited literature, several notes are made regarding issues that are commonly faced, and should be addressed when developing social robots for tele-presence applications, such as concerns regarding privacy, a proper control interface for the pilot (including a map of the environment and the robot’s surroundings), adaptability to people’s height and stance (e.g., sitting, standing, behind a desk), robustness towards communication failures (e.g., loss of WiFi connection), and dynamic volume control.
Finally, an important aspect of remote interaction is the translation of the operator’s input into robot behaviors. Many interfaces have been developed for controlling tele-presence robots, including graphical and tangible interfaces lazewatsky2011panorama, but also virtual reality tools nguyen2001virtual, or brain-machine interfaces tonin2011brain.
This category includes all interactions in which the robot and the human are located in a shared space and interact directly without explicit physical contact. This is the case for most existing social robotics scenarios.
Within these case we are most interested in mentioning the ones in which the robot has some form of locomotion ability (e.g., legged robot, aerial robots, wheeled robots), and also the ability to perceive and measure the distance to the human, in order to be able to actively control the distance between them. The social meaning of proximity in this context is referred to as proxemics, and constitutes an important part of non-verbal robot behavior mumm2011human.
Mead et al. mead2016perceptual have explored this topic by taking into account not only the psycho-physical and social aspects of proximity from the human’s perspective, but also regarding the robot’s needs. In terms of needs related to proximity, social robots may require or prefer certain distances to people in order for their sensors to work properly (e.g., vision, speech interaction).
Depending on the actual distance of the co-located robot, different modalities of communication may be more suitable. For example, robots in the private space may interact using speech or sound, and use touch screen for human input. However, robots at a greater distance but within line of sight, such as mobile robots, autonomous cars, or drones may use visual signals instead, such as expressive lights baraka2018mobile; szafir2015communicating.
Interactions happening in a shared space may involve an additional modality, namely physical contact between the human and the robot. Such interactions pertain to a blossoming subfield of HRI, commonly designated as Physical Human-Robot Interaction, or pHRI for short Haddadin2016; billard2013roboskin; youssefi2015skinware. From a hardware perspective, robots involved in pHRI are being designed with compliant joints (e.g., Baxter robot) for safety. Also, the design of robot outer shells is taking texture and feel into account yohanan2009tool. Moreover, novel paradigms for robot hardware are emerging with soft robotics majidi2014soft.
Examples of pHRI include physically assistive applications, where a robot has to be in physical contact with the person to execute its tasks, such as getting patients out of a chair shomin2015sit, or helping them feed song2012novel or dress themselves kapusta2016data. In industrial settings, physical proximity has also been shown, for some tasks, to improve the interaction and its perception by the workers huber2017developing.
On the other hand, physical contact may be used as a communication modality in itself, using a combination of touch, motion, pressure and/or vibration, known as haptic communication miyashita2007haptic. Such a communication modality is especially useful when others (e.g., visual) are not feasible. In particular, research has looked at how robots can communicate or guide people with visual impairments using physical contact. For example, Bonani et al. bonani2018my investigated the use of movement of a Baxter’s arm that blind people held to complement verbal instructions in a playful assembly task. Additionally, mobile robots have been used to guide people in indoor environments using physical contact kulyukin2004rfid; shomin2016navigation.
Moreover, physical contact may possess a social component. This is the case when a robot behavior utilizing physical contact with a human is meant to induce or influence their behavior. For example, a mobile robot may use physical contact when navigating through a human crowded environment, inducing people to move away shrestha2015using. Also, affective robot behaviors involving contact, such as a hug or a handshake, have been shown to have an influence on the social behavior of the humans in their interaction with the robot (e.g., self-disclosure or general perception of the robot) shiomi2017robot; avelino2018power. Human-robot haptics have also been investigated by studying the role of physical contact in human-animal interactions yohanan2012role.
While the spatial features discussed in this section pertain to different fields of research, one would expect in future robotic technologies a range of interactions that would incorporate a combination of the three, according to the task and situation at hand.
2.7 Temporal profile
In this section, we look at time-related aspects of interactions with a social robot. Knowing the intended temporal profile of these interactions may have a strong impact on the design of such robots. We specifically discuss the timespan, the duration, and the frequency of interactions.
Interactions with robots can be classified according to timespan, meaning the period of time in which the human is exposed to the robot. We consider four timespan categories, namely short-term, medium-term, long-term, and life-long. There does not exist, in the HRI literature, a quantitative way to establish the boundaries between these four categories, and as they may be context-dependent. Our aim is hence to provide a useful guideline for thinking about implications of such categories in the design of social robots, as well as their evaluation.
Short-term interactions typically consist of a single or only a few consecutive interactions, e.g., a robot giving directions in a mall. Of special importance for these types of interactions are design factors that influence the first impression of the human towards the robot (e.g., appearance, size, motion “at rest”, proxemics/approach behavior, initiation of the interaction). Usually very present in short-term interactions is the novelty effect, a fundamental characteristic of any innovation characterized by the newness or freshness of the innovation in the eyes of the adopter wells2010effect. It is a salient effect that plays a role in the adoption and use of novel media, characterized by higher initial achievements not because actual improvements occur, but due to the increased interest in technology clark1983reconsidering. This effect may help or harm the interaction depending on the its content and outcome, but it should be kept in mind in the design of robots for short-term use, also accounting for different expectations based on the users’ demographics.
Medium-term interactions go beyond a single or a few interaction(s) but do not extend over a timespan long enough to be considered part of the long-term category. They typically span several days or weeks. An example is a robot used to teach children a module in their curriculum over a few weeks. During repeated interactions, the novelty effect may wear off after the first few interactions, resulting in potential loss of interest or changes in attitudes towards robots over time gockley2005designing; kanda2004interactive. When considering repeated interactions with the same robot, it is hence essential to take this dynamic aspect into account by incrementally incorporating novelty or change in the behavior of the robot as well as maintaining a sense of continuity across interactions leite2013social; alves2019empathic. This will help sustain engagement and satisfaction both within and across individual interactions.
Long-term interactions include prolonged interactions that go beyond the period needed for the novelty effect to fade leite2013social. An example is a personal robot operating in a home. Long-term interactions typically create a sense of predictability in the human to know they will encounter a subsequent interaction. Additionally, humans may start feeling a sense of attachment to the robot, and even develop relationships with it. In addition to the points mentioned for the medium-term category, it is crucial to consider how the robot can both personalize and adapt its interactions with the human. Personalization means that the robot will accommodate for inter-individual differences, usually focusing on static or semi-static features of the human such as personality, preferences, or abilities. Adaptation means that the robot accommodates for intra-individual changes, focusing on dynamic features of the human such as physical, psychological and emotional state, performance, or behavior. For surveys about personalization and adaptation in HRI, please consult Rossi et al. rossi2017user and Ahmad et al. ahmad2017systematic. Personalization can also include a dynamic component; for example, an algorithm has been developed for an office robot to learn not only preferences of robot behaviors but also how to switch between them across interactions, according to personality traits of the human baraka2015adaptive.
Life-long interactions differ from long-term interactions by the fact that the human may go through large changes, for example, transitioning from childhood to adulthood, or progressively loosing some capabilities during old age. These types of interactions are much rarer with existing robots, but we do have examples that include robotic pets adopted in life-long timespans such as the AIBO or PARO robots. Another example is robots meant to accompany people until the end of their lives, such as robots assisting the elderly while gaining skills over time hence compensating for the decrease in their users’ capabilities georgiadis2016robotic. In the future, the vision of robotic companions dautenhahn2004robots may include richer interactions including mutual learning and evolution, emotional support, and building deeper bidirectional relationships.
Duration and frequency
In addition to timespan, an important temporal aspect of the interaction is the average duration of individual interactions. For example, a human can interact with a robot in short-term but prolonged interactions (e.g., in an educational context), or on the contrary in short interactions over a long timespan (e.g., office robot), or in other combinations and levels of the above. An important question to consider for longer durations is how to maintain engagement, especially with populations with a short attention span, such as children. For short durations, it is important to design for intuitiveness and efficiency of the interaction, in order to reduce the cognitive load or adaptation time of the human.
It is worth mentioning that duration is often imposed by the task itself, but may also be imposed by the human’s willingness to end it. For example, the Roboceptionist gockley2005designing
interacts with people in a building over large timespans. It was designed as a conversational chatbot, hence every person that interacts with it can initiate and end the interaction at any moment. The authors reported short interactions generally underseconds, and aimed at increasing this number by designing for long-term interactions with engagement in mind, using techniques from the field of drama.
In addition to timespan and duration, the frequency of interactions plays a role in their human perception by humans, and in the resulting design considerations. The frequency of interactions with the same robot can vary from very occasional (e.g., robots in stores visited sporadically) to multiple times per day (e.g., workplace robots). For high frequencies, a lack of of incorporation of novelty, or at least variation in the robot’s behavior, may result in fatigue and lack of engagement. Also, achieving continuity through memory is a particularly relevant factor leite2013social. Currently, the effect of frequency on the perception and effectiveness of interactions seems to be largely lacking in the HRI literature.
This concludes our discussion of time-related aspects of the interaction, as well as the discussion of our framework as a whole. Before concluding this chapter, we provide a brief discussion of design approaches for social robots.
3 Working within the social robot design space
The framework presented in this chapter outlined major dimensions of relevance to the understanding of existing social robots and the design of future ones. Moving forward, it effectively defines a design space for social robots, where each of the aspects discussed will involve a set of design decisions. For example: What role should my robot play in relation to humans? What should it look like? What kind of social capabilities should it have? What level of autonomy is best fitted for the task(s) and should it be fixed? etc. Higher-level decisions in the design process also arise such as: Are the requirements feasible with current technology, or will it require developing new technology? What are the practical considerations associated with the “theoretically best” design, as well as the costs, and are they outweighed by the benefits?
The actual design process of social robots and their interactions with humans has benefited from a number of design approaches inspired by design practices from a variety of fields such as engineering, computer science, HCI, and human factors. For example, some researchers in HRI have looked at developing design patterns that can be reused without having to start from scratch every time kahn2008design. There generally exist three broad design approaches, each of which may be valid depending on the intended context and objectives: human-centered design, robot-centered design, and symbiotic design. We briefly discuss these approaches next.
3.1 Robots as technology adapted to humans (human-centered design)
Human-centered design (HCD) is the central paradigms of HCI, and much of HRI design as a result. It aims to involve the intended user population as part of most development stages, including identifying needs and requirements, brainstorming, conceptualizing, creating solutions, testing, and refining prototypes through an iterative design process abras2004user.
In the HRI context, the main assumption is that humans have their own communication mechanisms and unconsciously expect robots to follow human social communication modalities, rules, conventions and protocols. Important aspects of the robot behavior and embodiment design that play a strong role in terms of the human’s perception of the interaction include physical presence bainbridge2008effect, size powers2007comparing, embodiment lee2006physically; wainer2007embodiment, affective behaviors leite2008emotional, role expectations dautenhahn2005robot, just to cite a few. From an evaluation point of view, HCD relies a lot on subjective self-reports of users to measure their perceptions, and complement more objective measures such as task performance.
While many HCD approaches exist for social robots, one of particular interest is treating robots as expressive characters, i.e., robots with the ability of expressing identity, emotion and intention during autonomous interaction with human users ribeiro2017. Designing for expressivity can be achieved for example by bringing professional animators to work side by side with robotic and AI programmers. The idea is to utilize concepts of animation developed over several decades ThomasJohnston1995 and apply them to robotic platforms breemen2004; takayama2011expressing; ribeiro2012; hoffman2012; gielniak2012; ribeiro2013.
3.2 Robots as goal-oriented technology (robot-centered design)
Historically, robots were developed solely by engineers who carried little concern about the human beyond the interface. While the focus in HRI has now shifted to a more human-centered approach as was discussed in the previous section, HCD as a general design paradigm has been criticized by many researchers who consider it to be harmful in some aspects greenberg2008usability; norman2005human. For example, it has been criticized for its focus on usability (how easy it is to use) as opposed to usefulness (what benefits it provides) and its focus on incremental contributions based on human input conditioned by current technologies, which prevents from pushing technological boundaries. Additionally, adapting the technology to the user may sometimes be more costly than having the user adapt to the technology.
As a result, there are cases where a more robot-centered approach may work best. Excessively adapting robots to humans may result in suboptimal performance, high cost of development, or unmatched expectations. It is important to recognize that in some cases, it may be better to ask the human to adapt to the robot (maybe through training) in order to achieve better performance on the long run. Humans have a much better ability to adapt than robots, and it is crucial to identify when robots should not adapt because it would be more efficient to ask or expect humans to do it norman2005human. In many cases, the robot may have needs that may incur an immediate cost on humans, but result in a better future performance. Examples include robots asking for help from humans when they face limitations veloso2015cobots, or teaching the robot to perform a certain task so that it can perform better in subsequent tasks. A robot-centered approach may also include the adaptation of our environments to make them suitable for robots. Examples include avoiding construction materials that are not compatible with the robot’s sensors, interfacing the robot with building facilities (such as elevators), and so on.
3.3 Robots as symbiotic embodied agents (symbiotic design)
Both approaches discussed above, whether human-centered or robot-centered, are valid approaches that one can use when designing social robots and their associated tasks. As a general design process for such robots, we advocate for the careful identification of strengths and weaknesses of each part and design for an increased symbiosis between the human(s) and the robot(s). One way to achieve this symbiosis is to adopt a holistic view that focuses on the overall system behavior, as a function of robot(s), human(s), and the environment steinfeld2009oz. For example, the CoBot robots are autonomous mobile robots veloso2015cobots servicing human users in a building, designed with the ability to utilize the presence of other humans in the environment (i.e., bypassers) to overcome their limitations. For instance, they ask for assistance in pressing the elevator button or putting objects in their basket since they do not have arms. This is an example of symbiotic autonomy where humans and robots service each other mutually in the same shared environment, and where both parties have to adapt to the other party’s needs.
In this chapter, we have introduced a framework for characterizing social robots and their interactions with humans along principal dimensions reflecting important design considerations. In particular, we (1) presented a broad classification of robot appearances, (2) repositioned existing classifications of robot social capabilities, (3) discussed a cross-section of purposes and application areas, (4) provided a straightforward and broad classification of the robot’s relational role, (5) clarified the related but distinct concepts of autonomy and intelligence, and discussed their quantification, (6) analyzed interactions according to their spatial features, and (7) looked at time-related aspects of the interactions. While this framework is aimed primarily at characterizing social robots by drawing from a large body of literature to illustrate the concepts discussed, it also serves as a useful guide to inform the design of future social robots. Towards this end, we briefly touched upon different design approaches, namely human-centered, robot-centered, and symbiotic.
Social robotics is a growing multidisciplinary field that brings closer aspects of human nature with aspects of robotic technology. The scope of what a social robot means, does, or serves, will be shaped by future developments in the field. In this journey towards creating interactive intelligent machines, we are hopeful that as they become more socially apt, they contribute to expanding, not reducing, the foundational aspects of our humanity.