Long-term memory allows humans to feel continuous and coherent in his/her thoughts, i.e., to be a continuous person with a continuous life. Hence, long-term memory is an essential component of social interaction between people in daily life, it allows remembering names, events, duties, relationships, etc. In fact, when long-term memory does not work properly due to illness (e.g., Alzheimer disease), the ability to interact with other human beings is severely damaged.
Therefore, it seems evident that a key aspect in achieving long-term interaction and social relationship between humans and social robots is the requirement of a long-term memory system for the latter. Memory constitutes an important part of a cognitive system implementation; in fact, it is the link of prior experiences with ongoing and future behavior. As stated by , a nontrivial level of social interaction requires that the robot should be able to use both semantic and episodic information. Both, semantic and episodic memories constitute the declarative long-term memory. While semantic memory is a repository for facts, such as knowing that the capital of Chile is Santiago, episodic memory is the memory of past experiences, i.e., remembering events and their associated context (places, persons, emotions, etc.). Both kinds of memories work together, and both need to be implemented for providing appropriate social skills for social robots. However, in contrast to implementing semantic memories, the implementation of episodic memories for social robots is much less developed, and therefore we focus our work on the latter.
The amount of information to be processed in a lifetime is vast; therefore, efficient methods are required for acquiring, filtering, encoding, storing, deleting and updating a robot’s episodic knowledge of its working environment. This information can be encoded in symbolic form and held in a storage module invoking the functionality of an episodic memory system. So far, this challenge has not been addressed appropriately, and the construction of long-term memories for social robots, which include both episodic and semantic components, has not been achieved. In this context, the main goals of this paper are: (i) to highlight the need of building appropriate long-term memory for social robots, (ii) to identify open questions that need to be answered in order to fulfill this first goal, and (iii) to provide basic concepts to address this challenge in the RoboCup@Home League with new tests. We propose the structure of such kind of tests, and give an example of a concrete test. This paper is organized as follows: The basic aspects of human memory are summarized in Section 2. Relevant open questions are identified in Section 3. The structure of the proposed tests for the RoboCup@Home League is presented in Section 4. Finally, some conclusions of this work are outlined in Section 5.
2 Human Memory
According to , memory is “the record of experience presented in the brain”. There are multiple memory systems that work in a complementary way. These systems have different functions, and they are characterized by different operating characteristics and brain structures in which they are embodied.
A first categorization is related to the persistency of the stored information. The Sensory Memory is able to store information acquired by our sensory systems just for the fraction of a second. This information then enters the so-called Short-Term Memory, which support “brief storage and immediate recall of substantial detail” . Part of the stored information is then consolidated into the Long-Term Memory. The term Working Memory is sometimes used instead of Short-Term Memory, although the working memory concept includes processes and structures used for the temporal storage and manipulation of information .
The Long-Term Memory is composed by a Declarative or Explicit Memory, which refers to information that is remembered consciously, and a non-Declarative or Implicit Memory, which refers to skills, abilities and tasks that can remembered implicitly (e.g., how to ride a bicycle). The Implicit Memory is further divided in Procedural Memory, in charge of storing procedures or ways of doing tasks, and Priming, which refers to the fact that some experiences are primed or recalled when a given stimuli is received. The Declarative Memory is divided into two complementary memories, the Episodic Memory and the Semantic Memory. Episodic memory, first defined by Tulving , refers to the memory of specific events occurring at a specific place and time and enables human beings to remember past experiences. Semantic memory is basically a repository for facts. Emotional experiences are also stored in the brain and they have influence on how other information is stored. The memory about emotions is stored in the declarative memory, while the emotional memory belongs to the implicit memory. Detailed explanations about the episodic, semantic and emotional memories can be found in [4, 7].
The main functionalities provided by the declarative long-term memory are: (i) the capability of remembering facts, concepts, events, experiences, skills, tasks, emotions, (ii) the ability to feel like a continuous person, (iii) a way of linking prior experiences with ongoing and future behavior, and (iv) the ability to interact with other human beings, i.e. be able to communicate coherently, and to build long-term relationships.
In this article our analysis will focus in the episodic memory, which as already mentioned, is related to the conscious remembrance of context-dependent events that are personally experienced. By context-dependent is it meant the cognitive state —the temporal, spatial and emotional/affective context —, as well as the embodied nature of the experience, i.e. the sensory-perceptual processing of a given experience . For instance, the experience/event of visiting your mother’s house for having dinner last night includes a temporal context (last night), a spatial context (your mother’s house), an emotional context (spending time nicely with your mother), and sensory-perceptual experiences (how the dinner tasted).
3 Open Questions for Implementing Long-Term Memory for Social Robots
During the past decade there have been several approaches that implemented episodic and semantic long-term memory in artificial systems (e.g., [5, 6, 10, 13, 14, 15, 16, 18, 22]). We believe that these works addressed only partially some of the main challenges that poses the implementation of long-term memory for social robots, and that still some of the questions described in the next paragraphs need to be answered.
How to store events and experiences in the form of episodes in the episodic memory?
It is still not clear how the cognitive state —temporal, spatial and emotional/affective context —associated to events, as well as the embodied nature of them, can be stored by a social robot. Naturally, the temporal context can be easily stored (e.g., using time stamps). The ability to determine and store spatial context has advanced largely in the last few years thanks to the deep learning revolution that allows the recognition of places, objects, and persons more easily. Yet, this has so far not been implemented in social robots that interact continuously with the changing world. Very few works have addressed the task of storing the emotional context and the embodied nature of the experience.
How to give different levels of relevance to the different episodes in terms of its novelty or the associate emotional state? As in the case of human beings, the stored episodes have different levels of relevance, which depends in the associate emotional state, among many other factors. Mechanisms for determining autonomously this level of relevance need to be developed.
How to store emotional states? In the human brain emotional situations are stored in the explicit and implicit memory systems, and experiences with a strong emotive content produce powerful and vivid memories. Mechanisms for implementing these functionalities need to be developed.
How to consolidate short-term memory into the long-term memory? The update of the long-term memory is a complex and time-consuming process that involves the consolidation of short-term memories into long-term ones . In the case of humans this is carried out during the sleep process, and consumes a large amount of brain resources . Therefore, the update of episodic information must be carefully designed and implemented in the robot case. No work has addressed the challenge of long-term memory consolidation for a social robot operating continuously in the real-word.
Which mechanisms to use for forgetting and repression? Given that the memory capacity is limited, forgetting mechanisms need to be implemented on basis of the relevance of the stored information. In addition, it must be analyzed if, as in the case of human beings, it is required to implement repression mechanisms that unconsciously block memories in order to protect the self from situations/emotions that she/he cannot cope with .
How to address the ethical issues related to the management of personal information of human beings acquired by social robots? A social robot with long-term memory will store information related with his/her human mates. It must be analyzed how this personal information will be protected, managed, and eventually, deleted.
4 Long-Term Memory in the RoboCup@Home League
In this Section we propose a new test for the RoboCup@Home league. First, we will describe the minimum requirements for an EpLTM (Episodic Long-Term Memory) implementation, how to validate them, and which sources of information are valid when generating memories. Finally, we present a test proposal for the competition, focused on EpLTM, Human-Robot Interaction (HRI) and perception.
At present, there is no consensus on the way an EpLTM should be implemented for service robots, but many approaches can be found [9, 10, 11, 12, 19, 20, 22]. As the RoboCup’s goal is to boost research, we propose to evaluate only the core requirements an EpLTM must fulfill, avoiding to force an specific implementation on the teams. The proposed requirements can be separated into 2 categories: I. exclusively episodic, and II. requirements related to historical and emotional relevances.
Category I is build from the 11 design requirements presented by Stachowicz . These were created to match the characteristics every EpLTM system must satisfy and are the minimum points for validation. The requirements (R1, R2, R4) declare that every episode must be recollected and stored by its spatio-temporal context: what, when, and where it happened. Moreover, there are no restrictions on which information the what field can contain; For the competition there are useful pieces of data to remember, for example, static information about known people or objects (name, age), and their dynamic state (last location, clothes, emotions). On the other hand, (R3, R6) state how the what field can be accessed and modified, while (R5, R7, R8) give some rules about the episode system structure (children episodes, anidation and transposition). In this work, R9 (non intrusiveness), R10 (efficiency), and R11 (scalability) are left out, because they relate to desirable design requirements and are not considered as candidates for validation during a test.
Category II adds the concept of relevance to each episode in memory, which is not covered by category I. Episodic relevance is essential when remembering interesting events, allowing access to episodes by their importance. On the one hand, we propose the historic relevance, which is directly related to the age of an episode; the lower its antiquity is, the higher its importance, which means a high probability to remember recent events. On the other hand, we propose the emotional relevance by assigning an emotion and its related magnitude to each episode; this allows to retrieve older but important events. It is important to stress that knowingwhy the episode is relevant and how the emotion relates to the episode is not required, as this depends on the emotion engine implementation.
4.2 Required information for validation
Next we present a proposal for the minimum data required to be stored for the competition, and the level of detail needed when validating the requirements (categories I and II). The concepts to be delimited are the episode definition, its contents (What, Where, When), and the associated emotions.
Although Stachowicz’s requirements do not impose the exact data to be gathered for (What, Where, When) fields, a set of verifiable entities should be defined for the competition. This serves as a way to normalize the validation process, by clarifying which data and format will be required for validation. It is important to highlight that by the following constraints we expect not to impose an implementation to the teams, but just to formalize the minimum required capabilities to any robot in competition.
In order to provide context to any episodic query or to enable a precise memory description by the robot, we propose at least the following nested episodic levels:
Context: RoboCup, Stage X, Test Y, Subtest Z. This let us identify generally the spatio-temporal context of an episode. Subtest Z only applies to tests where sub stages are clearly defined, as in: “Stage 1, Test: SPR, Subtest: The Riddle Game”.
Tasks: There must exist an episode related to each task or order executed by the robot. Tasks are defined inside a Context or inside other Task.
Capabilities: There must exist an episode related to each high-level robot capability: navigation, manipulation, perception, HRI. Capabilities are defined inside a Task.
Just knowing the sequence of episodes is not enough for validation. On the one hand, transposed episodes, i.e. episodes that are simultaneous, cannot be sequenced; on the other hand, the referees need a way to verify that the given episode description relates to the recorded time. Consequently, at least, the temporal information of an episode must consider initial and final timestamps. These can be described in terms of minutes, hours, days, weeks, months or years.
Location must be described in a simple way. Using coordinates like (x,y,z) is not allowed. What is allowed:
The robot can show a map of the arena with drawings marking the interesting locations.
The robot can describe locations using semantic information, with room names and elements of the arena. Some examples: Inside/Outside the arena, rooms (kitchen, bedroom), furniture (desk, fridge), or by using relative positions (at the left of, over the). For tests which require Simultaneous Localization and Mapping (SLAM) the description can be in terms of known areas, as the “bar” in the Restaurant test.
As the information to store is not clearly defined, this highly depends on the team implementation. However, at least some entities and fields should be defined by the referees/technical committee, in order to give a normalized base for all queries and validations during a test. Proposed entities to be stored are: people, objects and locations. Fields for each of these can be obtained from capabilities required on previous tests (e.g., age/emotion recognition, last seen location, and face images). Event description can be made verbally or by displaying a graph with the related sub-episodes.
The verbal description is preferred. This should be related as a story, by saying the associated episode sequence.
The description should be as specific as possible. E.g.: “I moved” vs. “I moved towards the door”.
Transposed episodes should only be considered if they are in context. These might require more complex verbal explanations to emphasize the concurrence of the actions.
How emotions are generated for a given robot and how they are associated to any given episode strongly depends on the emotional system used by the team.Therefore, no matter what emotional model is used, we propose to restrict the emotions to just 4 groups: Joy/Trust, Sadness/Fear, Surprise/Anticipation, Anger/Disgust. These are obtained from Plutchik’s theory of emotions . This selection is made to have a simple and verifiable set of emotions for the competition. Each episode must be related to at least one emotion and its intensity. Emotion intensity must have at least a resolution of 4 levels: “normal”, “a little <happy>”, “<happy>”, “very <happy>”.
4.2.6 Other limitations:
There are no proposed rules on how the memories are stored into semantic memory, as it depends on the implementation. There are no rules associated to other concepts.
4.3 Episode Generation
There are many sources from where the robot can generate episodes for the EpLTM: preparation for the competition, the travel, setup days, time between tests, non-RoboCup related episodes and the tests in which the robot participated. However, only the last one is a verifiable source of information. Then, we propose to limit the episodes to only the ones related to the tests of the current competition. On the first hand, this lets us simplify the evaluation and veracity of the memories. On the second hand, by only considering these episodes, new participants will compete in the same terms as older teams, this also serves as a regulation between the amount of memories gathered by teams which compete many times in a year.
As the competition lasts only a few days, the number of learning instances to gather episodes is small. In order to increase the amount of episodes, we propose two approaches. First and most important, the EpLTM test must be postponed as much as possible, ideally as the last test of Stage 2. The second proposal is to add a “Memory Setup” stage at the end of the “Setup Days” period, where each robot can generate interesting memories for the queries of the test. E.g.: By maintaining an informal conversation with someone of the committee or by an introduction to the people that it will find through the tests.
Depending on the queries the robot will encounter on the competition, the organizing committee can find new requirements. For instance for queries about people participating in 2 or more tests, the same name should be assigned to them, and they should wear similar but not identical clothes.
Finally, we have proposed to only consider episodes related to the current RoboCup competition, but it is important to mention that adding other events and older RoboCup competitions has some advantages. This directly affects the amount of episodes the robot will recollect. Moreover, this enables us to consider queries based on episodic inference, so the robot can be confronted to tasks requiring extrapolation from similar situations (e.g., Joe usually wants me to clean up the table after a meal). This capability can be added in later competitions, when the EpLTM test is considered solved.
The introduction of the memory concept is susceptible to cheating, for example, with manually written episodes during the competition or through a random episode generator. To attack this problem, first we consider the Fair Play concept. On the other hand, a strategy to hinder hardcoding is to increase the number of episodes and increase the number of available queries. However, the simple solution is to require evidence of each described episode.
Evidences fulfill two purposes, they hinder the cheating and also simplify the score assignation by the referees. When describing an episode it is desirable that the robot displays related evidence. As an example, the robot can provide a visualization as the one shown on Figure 1., where location, time, and context are given. Moreover, this can be displayed through the Vizbox  application, so that referees and audience can see it.
Task is validated by displaying the requirement as text.
Location is validated by displaying a colored map and images.
Sub-Episodes are validated with images or video.
4.5 Test Proposal: Sick and elderly care
Next we present an example of an @Home like test with EpLTM requirements. The test is based on the methodology proposed in this Section. Particularly, it is important to postpone the test as part of Stage 2, so that the robots can collect as much interesting episodes as possible beforehand. We expect that this proposal can be adjusted as needed for the competition.
The robot must help a sick or elderly person with reduced mobility (in bed/wheelchair), by answering questions about the home and recent events at which he cannot attend. The test is focused on EpLTM, perception and HRI.
The test takes place on the @Home arena. The operator is waiting in the bedroom, lying on the bed or sitting on a chair. The arena keeps the same structure and items as in other tests, but with some small changes. Other people are located in the house with which the robot can interact as needed.
The robot starts by entering the arena, it moves to the bedroom, approaches the operator and asks if any assistance is needed. The operator explains he is sick/tired and cannot move, so it will make some questions to the robot. After 4 queries, the operator tells the robot to leave the bedroom.
Queries can be separated into 3 categories, depending on their requirements. The robot should answer at least one question of each category.
Cat 1: Queries about memories and emotions.
Cat 2: Queries which require investigating objects in the arena.
Cat 3: Queries which require interacting with people in the arena.
We recommend the use of an episodic queries generator, built to match requirements of categories I and II.
The robot must show evidences when answering, e.g., in a screen or using Vizbox , as shown on the official Rulebook.
Referees must validate the coherence between answers and provided evidence.
Rooms, people and objects must be set up according to the possible queries.
The development of EpLTM for social robots is an important challenge for improving human-robot interaction, and @Home has the means to boost the progress. For that reason, we have presented a test proposal focused on EpLTM, its requirements and validation strategies. The proposal is focused on the minimum requirements any social robot implementing EpLTM should fulfill, but trying not to impose an specific implementation to the teams.
-  Allen, P.A., Kaut, K.P., Lord, R.R.: Chapter 1.8 emotion and episodic memory. In: Dere, E., Easton, A., Nadel, L., Huston, J.P. (eds.) Handbook of Episodic Memory, Handbook of Behavioral Neuroscience, vol. 18, pp. 115 – 132. Elsevier (2008)
-  Bailey, C.H., Bartsch, D., Kandel, E.R.: Toward a molecular definition of long-term memory storage. Proc. Natl. Acad. Sci. U.S.A. 93(24), 13445–13452 (Nov 1996)
-  van Beek, L., Chen, K., Holz, D., Sanchez, L., Matamoros, M., Nagano, H., Rascon, C., de Souza, J.R., Rudinac, M., Wachsmuth, S.: RoboCup@Home 2017: Rules and Regulations. http://www.robocupathome.org/rules/2017_rulebook.pdf (2017)
-  Dere, E., Easton, A., Nadel, L., Huston, J.P.: Handbook of behavioral neuroscience. In: Handbook of Episodic Memory, Handbook of Behavioral Neuroscience, vol. 18, p. iii. Elsevier (2008)
-  Deutsch, T., Gruber, A., Lang, R., Velik, R.: Episodic memory for autonomous agents. In: 2008 Conference on Human System Interactions. pp. 621–626 (May 2008)
-  Dodd, W., Gutierrez, R.: The role of episodic memory and emotion in a cognitive robot. Proceedings - IEEE International Workshop on Robot and Human Interactive Communication 2005, 692–697 (2005)
-  Eichenbaum, H.: Learning & Memory. W. W. Norton & Company (2008)
-  Eichenbaum, H.: Memory. Scholarpedia 3(3), 1747 (2008)
-  Ho, W.C., Dautenhahn, K., Lim, M.Y., Vargas, P.A., Aylett, R., Enz, S.: An initial memory model for virtual and robot companions supporting migration and long-term interaction. In: RO-MAN 2009 - The 18th IEEE International Symposium on Robot and Human Interactive Communication. pp. 277–284 (Sept 2009)
Jockel, S., Weser, M.: Towards an episodic memory for cognitive robots. European Conference on Artificial Intelligence pp. 68–74 (2008)
-  Kelley, T.D.: Robotic Dreams: A Computational Justification for the Post-Hoc Processing of Episodic Memories. International Journal of Machine Consciousness 06(02), 109–123 (2014)
-  Kim, M.J., Baek, S.H., Cho, S.H., Kim, J.H.: Approach to integrate episodic memory into cogency-based behavior planner for robots. In: 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC). pp. 4188–4193 (oct 2016)
-  Kuppuswamy, N.S., Cho, S.H., Kim, J.H.: A cognitive control architecture for an artificial creature using episodic memory. 2006 SICE-ICASE International Joint Conference pp. 3104–3110 (2006)
-  Laird, J.E., Newell, A., Rosenbloom, P.S.: Soar: An architecture for general intelligence. Artificial Intelligence 33(1), 1 – 64 (1987)
-  Leconte, F., Ferland, F., Michaud, F.: Design and integration of a spatio-temporal memory with emotional influences to categorize and recall the experiences of an autonomous mobile robot. Auton. Robots 40(5), 831–848 (Jun 2016), http://dx.doi.org/10.1007/s10514-015-9496-2
-  Nuxoll, A., Laird, J.E.: A cognitive model of episodic memory integrated with a general cognitive architecture. In: ICCM (2004)
-  Plutchik, R., Kellerman, H.: Emotion: Theory, research, and experience. In: Theories of Emotion, p. ii. Academic Press (1980)
-  Ratanaswasd, P., Gordon, S., Dodd, W.: Cognitive control for robot task execution. In: ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005. pp. 440–445 (Aug 2005)
-  Spexard, T.P., Siepmann, F., Sagerer, G.: A Memory-based Software Integration for Development in Autonomous Robotics. In: International Conference on Intelligent Autonomous Systems. pp. 49–53 (2008)
-  Stachowicz, D., Kruijff, G.J.M.: Episodic-like memory for cognitive robots. IEEE Transactions on Autonomous Mental Development (2012)
-  Tulving, E.: Episodic and semantic memory. Organization of Memory (Tulving E, Donaldson W, eds). pp. 381–403. Academic Press, New York. (1972)
-  Vijayakumar, S.: Long-Term Memory in Cognitive Robots. Ph.D. thesis, Universitaet des Saarlandes (2014)
Walker, M.P., Stickgold, R.: Sleep-dependent learning and memory consolidation. Neuron 44(1), 121–133 (Sep 2004)
-  Wood, R., Baxter, P., Belpaeme, T.: A review of long-term memory in natural and synthetic systems. Adaptive Behavior 20(2), 81–103 (2012)