Detection of Human Contact by Thermal Exchange Observer of Robot Skin

05/05/2020 ∙ by Yukiko Osawa, et al. ∙ 0

Physical human-robot interaction (pHRI) enables to achieve various assistive tasks for people by robots such as collaborative robotics or motion assistance. It pertains both planning-control and hardware considerations. We propose a new soft robot skin that can detect human contact through thermal features. An electrically conductive pipe is interposed between graphite sheet that has high heat conductivity and soft material for cover compliance. Then the temperature of the water pipe is controlled by a heat source and a small water pump. Besides, an observer for estimating heat flow is implemented; it can detect human interaction by measuring the temperature change of the water pipe. Hence, there is no need to attach sensors to the robot skin's surface, which can be kept soft and eventually textured with a desired pattern. We assessed the validity of the prototype in experiments of contact detection by human fingers.



There are no comments yet.


page 1

page 2

page 3

page 4

page 5

page 6

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Physical interaction between human and robot enables to achieve various robot tasks: helping a frail person in daily life, programming a robot in manufacturing through touch guidance and demonstration, etc. In order to achieve such tasks like a human, robots need the physical features such as soft texture and warmth giving from human skin, which every human has, for providing pleasantness to touch and a kind of a secure feeling. Therefore, thermal exchanges and softness or touch material properties of the robot cover play an important role in embodiment, acceptability, and human friendliness. We claimed that soft material covering a robot –called robot skin, that has the thermal capabilities is one of the solutions for promoting pHRI.

Indeed, human-robot physical interaction has several positive effects, especially in caring for humans. For example, patients who are touched by a nurse before their surgical operation have a lower stress level [1]. The study in [2] shows that a gentle stroking touch lower blood pressure and increase transient sympathetic reflexes. The feeling of pleasantness generated from physical interaction has been analyzed in neuroscience and psychological fields. The study suggests the correlation between stimulus frequency and pleasantness [3]. Besides, contact force of a finger sliding has been analyzed in the spatial domain [4]. In particular, warmth in physical interactions gives us beneficial effects. The literatures [5, 6] show physical interactions with warm mediated touch devices generated in a stronger sense of trust and friendship. Therefore, texture, softness, and temperature on the contact surface affects human feeling. In this domain, we focus our investigations on combining a cover that gathers both thermal exchanges and softness. In [7] we propose a cover soft cover technology with thermal display capabilities. In this paper, we address the other aspect that is of prime importance in human-robot pHRI: the ability for the robot to detect human touch. The novelty of our approach w.r.t existing state of the art contact detection, is in using solely thermal exchange observers through the robot cover.

Fig. 1: Prototype of the robot skin.

Ii Related work

There are several existing methods for detecting human contact by using various sensors technology. In fact, all existing artificial robot skins are made exactly for that purpose! A large coverage of tactile sensing technologies is thoroughly reported in [8, 9]. This area of research is rich, with extensive different transduction technologies ranging depending on the robotic application requirements. There are also tactile sensing technologies based on printed electronics techniques [10], including deformable and even stretchable support and transduction. Such technologies allows more sophisticated tactile sensing that goes beyond touch, and not all of them embed also thermal exchange sensing as part of the skin.

There are also a class of different approaches for robot touch detection that do not require a dedicated skin and uses existing internal robot sensors that serve other purpose. The excellent survey by [11] summarized such methods of robotic contact sensing. Such method relay in principle on an observer that can reliably detect contact through expected versus actual state residuals. For example, [12] proposed the momentum observer, that has been extended even for low-cost personal robots, considering non-linear effects such as backlash and friction [13]. Such methods can perform even better when combined with existing (if any) robot terminal points force sensing [14, 15] in multi-contact. In [16] a contact detection method uses only joint encoders and learned robot motion dynamics.

Our aim is to achieve robot covers that are soft and capable of thermal display [7]. Our idea is to exploit the thermal cover duality in both displaying and detecting of human touch through monitoring the thermal exchange in the process of robot skin thermal regulation. To our best knowledge, this idea is totally new.

There are many kinds of thermal displays based on a Peltier pump heat source that can either heat or cool the surface based on the Peltier effect [17], see e.g. in haptics [18, 19, 20]. Our soft robot skin uses thermal transfer through water flow [7], and our idea builds on the structural design of our robot skin.

Our paper proposes a method to allow our robot skin to also detect human contact by estimating heat flow on its surface. Our main contributions are specifically as follows:

  • with respect to [7] we considered a better thermally conductive outer layer and water-pipe to compose our soft robot skin;

  • we control directly the water pipe temperature from the heat source (Peltier pump as in [7]) attached to a small water tank, and show that human contact can be detected through thermal features;

  • there is no sensor on the robot outer skin; hence its surface can be kept soft, thermal conductive and can be patterned with different texture serving human touch requirements and eventually easthetics.

Figure 1 shows the developed prototype of the robot skin. An electrically conductive pipe, different from that of [7] is interposed between the graphite sheet (the new outer-layer) and the soft material. In this paper, the temperature of the water pipe is controlled directly by a heat source and a small water pump. Besides, an observer for estimating heat flow is implemented for detecting human interaction without using sensor on the surface of the prototype. We assessed the validity of our idea through a prototype of the robot skin in experiments of contact detection by human fingers.

Fig. 2: Proposed robot-skin system.

Iii Design structure of the robot skin

Figure 1 shows the developed prototype of the robot skin, which size for our study investigations is 30 mm 10 mm. This is because a free sample come at this dimension. This prototype is made to assess our proposed human touch detection method with a revisited version of our robot skin in [7]. For circulating water inside of the robot skin, we use another water pipe with an inner diameter of 1.9 mm. This pipe is made of nickel and copper, and was chosen for its high thermal conductivity. It is interposed between a graphite sheet and a foam material. The graphite material having a relatively high thermal conductivity, is used instead and in place of skin’s upper layer composite material in [7]). Its role is to display a user-programmed or context-defined thermal behavior of the robot. The foam material composing the compliant part of the skin is similar to [7]. The thickness of the sheet and the foam layer is 0.5 mm and 5 mm, respectively. A silicone pipe with inner diameter of 3.0 mm is used for connecting the water pump and the water tank, and the value of the diameter is used for calculating the water-flow model as a characteristic diameter.

Our system is shown in Fig. 2. By varying the temperature of the Peltier pump connected to the water tank, the temperature of the water pipe is controlled to the desired temperature. At the same time, human contact is detected by monitoring the temperature change of ; it means that any sensor on the surface of the prototype is not needed. In this paper, we refer to this system as “the robot-skin system”. The proposed detection method is explained in the next section.

Iv Proposed detection method

Fig. 3: Modeling the prototype of the robot skin.

Iv-a Modeling of the robot-skin system

Thermal systems can be modeled based on a thermal network method where thermal properties are expressed as electric circuits analogies. The Fig. 3 shows the thermal model adopted for the prototype of robot skin, where , , , , , , , , and stand for temperature of the water pipe, temperature of the graphite sheet, thermal capacitance of water pipe and graphite sheet, thermal resistance between the pipe and the sheet, heat flow from the Peltier pump, heat flow from human finger, and heat dissipation of the air at the pipe and the sheet, respectively. To apply the model to the thermal network method, thermal resistance and thermal capacitance of the water pipe are defined as


where , , , , , and stand for thermal resistance of the water pipe, convection heat-transfer coefficient, specific heat and density of water, area and volume of the water tank, respectively. Here, and are derived as


where , , and denote ambient temperature, thermal resistance of air at the water pipe and the graphite sheet, respectively.

The value of switches depending on the case of water flowing or not; this is because the thermal phenomenon changes heat transfer to pure heat conduction through the water when water flow stops. Base on the empirical relationship for laminar flow [21], heat transfer coefficient is derived as

where , , , , , and Pr stand for thermal conductivity, diameter of the pipe, length of the pipe and the tank, density, viscosity, velocity of water flow, and Prandtl number, respectively.

The block diagram of the robot-skin system is shown in Fig. 4. Based on this figure, the transfer function of the robot-skin system is obtained as


where is the Laplace operator. This model includes two thermal capacitances and the structure is similar to two inertia system in a mechanical system too. An observer for estimating heat flow from human finger (our any external touch) is calculated based on this thermal model.

Fig. 4: Block diagram of the robot-skin system.
Fig. 5: Block diagram of the thermal exchange observer.
Fig. 6: Control algorithm of the robot-skin system.

Iv-B Thermal exchange observer

To detect the thermal response of the robot’s skin changed by human interaction, heat flow needs to be monitored. Heat flux sensor can obtain a heat flow response easily, but it biases heat conduction from a heat source and is difficult to design it soft on a surface. In this paper, heat flow from a human finger is estimated by temperature information obtained by the thermocouple attached to the electrically conductive pipe. For sensing human interaction, an observer for estimating heat flow shall be implemented. The algorithm of this observer is based on a disturbance observer [22] that has a useful structure and can apply in many fields of control systems. Indeed, this observer can estimate heat flow in the load side of a control system and is suitable for this case. Therefore, the observer is applied to the prototype and called “thermal exchange observer” in this paper. From Fig. 4, the transfer function from to can be calculated as


Here, the transfer function was calculated except and the effect of adds to the estimated value derived from the observer at the end of the calculation. Therefore, is expressed by using the thermal model as


Based on eq. (7), the observer for estimating is derived as


Here, denotes the estimated value. By using eq. (8), a structure of the observer is expressed as shown in Fig. 5.

V Temperature control system

Figure 6 shows whole control system. The temperature of the flowing water is controlled by the Peltier pump attached to the water tank. Since the Peltier pump has a nonlinear behavior [7], a discrete-time autoregressive moving average (ARMA) model is used [23] for temperature controlling of the Peltier pump. As for the controller, a model preview controller (MPC) is also used to regulate the temperature of the robot-skin system. The cost function with the constraints are calculated as:


subject to


where , , , , , , , and stand for the discrete time, prediction horizon, weight values of the cost function, sampling time, temperature command of , and minimum and maximum value of threshold of the calculated input, respectively. Here, temperature response of the Peltier pump is the input of the MPC. The first term of (9) works for tracking the temperature of the water pipe to its temperature command , and the second one is for suppressing rapid change of the input . In the case where the temperature response is larger than its temperature command, the velocity of water flow turns to 0 m/s for stopping heat transfer from the Peltier pump. It is possible to cool water by cooling the Peltier pump, but it takes time because the thermal capacitance of the water is too large. Therefore, water is cooled using natural heat loss after the response achieving the desired temperature.

As for contact detection, the thermal observer can be based only on the heat flow from the human finger, and the estimated value is not influenced by the temperature control. It means the detection of human contact while controlling the temperature is possible using our proposed detection system.

Fig. 7: Experimental setup.
Parameter Description Value
sampling time 1.0 ms
Thermal conductivity of water 0.1 m W/mK
Specific heat of water 4.208 kJK/kg
Density of water 999.8 kg/
Viscosity of water 1.55 kg/ms
Diameter of the pipe 3.0 m
Length of the pipe 0.11 m
Area of water tank 3.2
Volume of water tank 3.2
Pr Prandtl number 11.35
Thermal capacitance of water pipe 0.134 J/K
Thermal capacitance of graphite sheet 0.0124 J/K
Thermal resistance of graphite sheet 10.0 K/W
Thermal resistance of air at water pipe 2100 K/W
Thermal resistance of air at graphite sheet 1.0 K/W
Minimum value of threshold of 7 C
Maximum value of threshold of 37 C
Desired temperature command 24–28 C
Prediction horizon 100
Weight value of eq. (9) 1.0
Weight value of eq. (9) 0.03
Proportional gain 0.1
Integral gain 0.03
TABLE I: Parameters for experiments.
Fig. 8: Experimental results of temperature control.

Fig. 9: One finger contact.
Fig. 10: Two fingers contact.
Fig. 11: Three fingers contact.

Fig. 12: Temperature responses of one finger contact.
Fig. 13: Temperature responses of two fingers contact.
Fig. 14: Temperature responses of three fingers contact.

Fig. 15: Estimated heat flow of one finger contact.
Fig. 16: Estimated heat flow of two fingers contact.
Fig. 17: Estimated heat flow of three fingers contact.
Fig. 18: Estimated heat flow of three persons data.
Fig. 19: Estimated heat flow comparing finger positions.

Vi Experiments

Vi-a Outline

Fig. 7 shows the experimental setup. Two Peltier pumps 40mm 40mm were used for heating the water tank 40mm 80mm, and the heat sink and the fan was attached to the Peltier pumps at back sides for heat radiation. To measure the temperature, the thermocouples were attached to the Peltier pumps and the electrically conductive pipe of the prototype. There are two kinds of experiments. First, the temperature controller was verified. The temperature command was set to 24, 26, and 28 C, and was controlled based on its command derived from the MPC. The velocity of the water pump keeps 0.141 m/s, except for the case where is larger than its command. Second, the detection method was verified. The temperature command was set to 24C, and heat flow from human fingers was estimated by the thermal exchange observer explained in Sec. IV-B. Since there is no sensor on the surface of the graphite sheet and can not be obtained, was calculated using instead of in the experiments. For investigating the relationship between the heat flow and contact area/position, two cases of detection tests were conducted. In the first case, the estimated heat flows were compared in the case of contacting one/two/three fingers. The estimation results are compared to the data from three persons: age is 27–28 years old (man1, woman1, woman2). As for the second case, the values were compared in different contact positions; case1 (near the thermocouples), case2 (middle), and case3 (far from the thermocouples). In these experiments, the human finger contacted the prototype every 3 seconds. The parameters for the experiments were summarized in Table I.

Vi-B Experimental results

Experimental result of temperature control is shown in Fig. 8, the temperature dynamic change is known to be slow. It shows that the temperature of the water pipe tracked to its temperature command. The responses exhibits chattering sometimes around the desired temperature because the water pump repeatedly switch on and off when the response reaches its desired valued. Experimental results of detection for one/two/three fingers are shown in Figs. 17 to 17. In every cases, temperature of the pipe was controlled to 24 C, other than the time that human contacted the prototype. From Figs. 17, 17, and 17, the estimated heat flows from human fingers tend to increase alongside the larger contact area (number of fingers). The three persons results were summarized in Fig. 18, and these results exhibits similar tendency although variations depending on persons can be observed. Figure 19 shows the estimated heat flow in the case 1, 2, and 3. The figure shows that heat flow from a finger is also changed depending on the contact positions. This is because thermal attenuation occurs depending on the distance between contact position and a thermocouples. The theoretical explanation about this is mentioned in [24, 25]. Therefore, the estimated heat flows were influenced by the contact area and positions. Since the prototype is small in size and it could be verified only contacting one to three fingers, other contact taxonomies such as grasping, holding, and stroking will be tested in future work with a integrated whole-scale system.

Vii Conclusion and Future work

This paper proposed a new soft robot skin that can at a time display thermal behavior of the robot, and still detect human contact through transient thermal exchange monitoring when touched by a human. For detecting human interaction, the thermal exchange observer was implemented in the control system; it works by estimating heat flow, which value changes depending on the temperature of the water pipe. Therefore, the detection method does not need to measure temperature on the robot skin, and its surface can be keep soft and patterned at will. The validity of the proposed system was confirmed by some experimental results. For future work, we aim to extend our algorithm to detect multi-point contact and identify the nature of touch beyond human through the signature of the heat flow dynamics and softness. Moreover, the value of the estimated heat flow will be utilized to detect precise human interaction, such as the grasping area, strength, and human body temperature. We will also improve a heating and cooling system to be more effective, and will cover important areas of the humanoid robot by the soft thermal robot skin.


This work was supported in part by JSPS Overseas Research Fellowships No. 201960463.


  • [1] S. J. Whitcher and J. D. Fisher, “Multidimensional reaction to therapeutic touch in a hospital setting,” Journal of Personality and Social Psychology, vol. 37, no. 1, pp. 87–96, 1979.
  • [2] H. Olausson, J. Cole, K. Rylander, F. McGlone, Y. Lamarre, B. G. Wallin, H. Krämer, J. Wessberg, M. Elam, M. C. Bushnell, and Å. Vallbo, “Functional role of unmyelinated tactile afferents in human hairy skin: sympathetic response and perceptual localization,” Experimental Brain Research, vol. 184, no. 1, pp. 135–140, Jan 2008.
  • [3] R. Ackerley, I. Carlsson, H. Wester, H. Olausson, and H. Backlund Wasling, “Touch perceptions across skin sites: differences between sensitivity, direction discrimination and pleasantness,” Frontiers in Behavioral Neuroscience, vol. 8, p. 54, 2014. [Online]. Available:
  • [4] M. Wiertlewski, C. Hudin, and V. Hayward, “On the 1/f noise and non-integer harmonic decay of the interaction of a finger sliding on flat and sinusoidal surfaces,” in IEEE World Haptics Conference, June 2011, pp. 25–30.
  • [5] J. Nie, M. Park, A. L. Marin, and S. S. Sundar, “Can you hold my hand? physical warmth in human-robot interaction,” in ACM/IEEE International Conference on Human-Robot Interaction, March 2012, pp. 201–202.
  • [6] E. Park and J. Lee, “I am a warm robot: the effects of temperature in physical human-robot interaction,” Robotica, vol. 32, no. 1, pp. 133–142, 2014.
  • [7] Y. Osawa and A. Kheddar, “A Novel Robot Soft Cover with Thermal Display Capabilities,” Feb. 2020, working paper or preprint. [Online]. Available:
  • [8] B. D. Argall and A. G. Billard, “A survey of tactile human-robot interactions,” Robotics and Autonomous Systems, vol. 58, no. 10, pp. 1159–1176, 2010.
  • [9] R. S. Dahiya and M. Valle, Robotic tactile sensing: technologies and system.   Springer Science & Business Media, 2012.
  • [10] S. Khan, L. Lorenzelli, and R. S. Dahiya, “Technologies for printing sensors and electronics over large flexible substrates: A review,” IEEE Sensors Journal, vol. 15, no. 6, pp. 3164–3185, 2015.
  • [11] S. Haddadin, A. De Luca, and A. Albu-Schffer, “Robot collisions: A survey on detection, isolation, and identification,” IEEE Transactions on Robotics, vol. 33, no. 6, pp. 1292–1312, 2017.
  • [12] A. de Luca and R. Mattone, “Sensorless robot collision detection and hybrid force/motion control,” in IEEE International Conference on Robotics and Automation, 2005, pp. 999–1004.
  • [13] F. Flacco and A. Kheddar, “Contact detection and physical interaction for low cost personal robots,” in IEEE International Symposium on Robot and Human Interactive Communication, 2017, pp. 495–501.
  • [14]

    G. Buondonno and A. De Luca, “Combining real and virtual sensors for measuring interaction forces and moments acting on a robot,” in

    IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016, pp. 794–800.
  • [15] L. Manuelli and R. Tedrake, “Localizing external contact using proprioceptive sensors: The contact particle filter,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, 2016, pp. 5062–5069.
  • [16] A. Bolotnikova, S. Courtois, and A. Kheddar, “Contact observer for humanoid robot pepper based on tracking joint position discrepancies,” in IEEE International Symposium on Robot and Human Interactive Communication, 2018, pp. 29–34.
  • [17] Y. G. Gurevich and J. E. Velazquez-Perez, Peltier effect in semiconductors.   American Cancer Society, 2014.
  • [18] L. A. Jones and H. Ho, “Warm or cool, large or small? the challenge of thermal displays,” IEEE Transactions on Haptics, vol. 1, no. 1, pp. 53–70, Jan 2008.
  • [19] M. Guiatni and A. Kheddar, “Modeling Identification and Control of Peltier Thermoelectic Modules for Telepresence,” Journal of Dynamic Systems, Measurement, and Control, vol. 133, no. 3, March 2011.
  • [20] A. Manasrah, N. Crane, R. Guldiken, and K. B. Reed, “Perceived cooling using asymmetrically-applied hot and cold stimuli,” IEEE Transactions on Haptics, vol. 10, no. 1, pp. 75–83, Jan 2017.
  • [21] J. P. Holman, Heat transfer.   McGraw-Hill, Inc, New York, 1990, vol. 1.
  • [22] K. Ohnishi, M. Shibata, and T. Murakami, “Motion control for advanced mechatronics,” IEEE/ASME Transactions on Mechatronics, vol. 1, no. 1, pp. 56–67, 1996.
  • [23] M. Guiatni, A. Drif, and A. Kheddar, “Thermoelectric modules: Recursive non-linear arma modeling, identification and robust control,” in Annual Conference of the IEEE Industrial Electronics Society, Nov 2007, pp. 568–573.
  • [24] Y. Osawa and S. Katsura, “Thermal propagation control using a thermal diffusion equation,” IEEE Transactions on Industrial Electronics, vol. 65, no. 11, pp. 8809–8817, Nov 2018.
  • [25] Y. Osawa and S. Katsura, “Sensing of heat source in a deep layer by considering heat propagation,” IEEJ Journal of Industry Applications, vol. 7, no. 3, pp. 229–235, 2018.