Low fertility rates and progressive lengthening of life expectancy keep exacerbating the population aging problem among the developed and even developing countries [WB:2019]. A growing demand emerges for the need for elderly care for those who have difficulties in daily activities. Literature survey shows that social robots as a companion provide a positive influence with reduced stress and an improved sense of security among people with aging and dementia [Alonso2019Social]. Robotic solutions for the elderly need to take both physical and cognitive consideration into the design process of intelligent and interactive agents [Cynthia2019Designing]. While the elderly are becoming more open to social robotic solutions, it remains a challenge to integrate novel solutions such as wearable technologies with active assistance for elderly users.
The sit-to-stand movement is closely related to the falls of the elderly, which is the leading cause of injury and the primary etiology of accidental deaths among the elderly over 65 years of age, which accounts for 70 percent of accidental deaths in persons 75 years of age and older [Tinetti2003Preventing]. The sit-to-stand test is a widely adopted metric for activities of daily lives (ADLs) to measure the necessary mobility skills of the elderly [Podsiadlo1991Timed, Beauchet2011Timed]. It is commonly practiced when trying to leave the seat, such as the bed, chair, or toilet, during the transition between sitting and walking [Isaacs1985Clinical].
The supernumerary robotic limbs (SuperLimbs) emerge in recent research exploring the cognitive boundary of brain-machine interaction for wearable robotics [Asada2017Independent], [Gonzalez2018Design]. Much of the SuperLimbs research focuses on healthy subjects with normal brain and motor functions, with SuperLimbs as an augmentation system for collaborative assistance in certain work conditions [Gonzalez2019Loop]. However, there remains a research gap exploring the potential daily assistance of SuperLimbs for elderly users with degraded brain and motor functions in the ADLs, which is the focus of this paper. In particular, we propose the design of a robotic cane system with pneumatic actuation, inflatable vest, and privacy-safe range sensing to explore the use of SuperLimbs for elderly assistive care during the sit-to-stand transition at home, shown in Fig. 1.
I-a Related Work
I-A1 The Sit-to-Stand Motion Transition
The literature classifies the sit-to-stand movement into three phases, namely trunk flexion, hip lift-off, and the knee-hip extension[Millington1992Biomechanical]. Trunk flexion results in forward shifting, transforming the support to only feet [Hughes1994Chair]. Hip lift-off is the key event in transforming the forward motion in an upward direction. After that, the maximum vertical ground reaction force occurs. Knee-hip extension lifts the body weight in the vertical direction until full standing [Scarborough2007Chair].
I-A2 Robot Design for the Elderly
Existing literature on the robot design guidelines for the elderly is mainly focused on social robots for cognitive care, such as stress relief and mental support [Broekens2009Assistive]. While it is generally accepted that robot designs should take the user needs of the elderly into considerations, yet the task is still more complicated in practice [Pape2002Shaping]
. The adoption of agents of artificial intelligence is considered as a potential solution to address the individual characteristics of the elderly user[Cynthia2019Designing], but further developed is required to integrate these agents with active physical support for elderly usage.
I-A3 Motion Detection for Elderly Care
The traditional elderly care mainly depends on human labor, which is costly. The researchers have been searching for effective methods to offer autonomous and round-the-clock caring for the growing aging communities. Some wearable devices installed with biosensors and inertial sensors have been developed for the elderly health-monitoring [pantelopoulos2009survey] and activity recognition [avci2010activity]. However, these intrusive wearable devices need to be worn all day round. Due to the limitations of measuring range, users may need to wear multiple sensors when a large range of daily activities are monitored [chen2017survey].
On the other hand, the researchers have been actively searching for non-intrusive approaches to detect the daily life activities with minimal effect to the users. RGB-video based action recognition provides an economical solution [foroughi2008intelligent, rougier2011fall]. However, since this method requires the RGB-video sequence to be recorded, it has the potential of violating the user’s privacy. With the emergence of a low-cost depth sensor, the researchers have begun to utilize depth information for elderly activity detection [presti20163d]. Depth cameras, in particular, provide cost-effective real-time 3D body movement data. Without recording the raw depth-image, depth cameras, which only records the desired body joint data, is privacy-preserving, comparing to the previous approaches. Furthermore, the vision system indicates the potential to empower the intelligent around-the-clock ambient interaction in the elderly healthcare with little labor cost, as demonstrated in [YeungA] and [3d_point_cloud_icu_care]. With the unprecedented development of 3D technology, the depth-sensor will be able to detect various activities and serves for other purposes, such as fall prevention and long-term health monitoring.
I-B Proposed Method and Contributions
In this paper, we propose a soft robotic technique to design a robotic cane system, providing sit-to-stand assistance for the elderly usage at home. Through the integration of a pneumatically-driven linear actuator for active lower-limb support, an inflatable vest for compliant interaction between the human body and the robotic cane, and a privacy-safe depth vision for intention detection to compensate for potential issues with cognitive usage by the elderly. The contributions of this paper are listed as the following.
The design and development of a robotic cane system as a supernumerary robotic limb for the elderly assistance;
The integration of soft and non-intrusive robotic techniques for active elderly assistance in forms of pneumatic actuation, inflatable vest, and privacy-safe sensing;
A modified biomechanical model of sit-to-stand movement with an active cane for analysis and control with experimental verification;
The experimental validation of each component of the proposed robotic cane system;
Preliminary design guidelines of supernumerary robotic limb system for elderly care.
The rest of this paper is organized as follows. Section II formulates the problem of sit-to-stand transition, followed by the proposal of a modified biomechanical model with a robotic cane attached. In this section, we also introduce the system design of the supernumerary robotic limb system of the robotic cane, inflatable vest, and privacy-safe camera. Section III presents the experiment setup, procedure, and results to validate the feasibility of such a robotic cane for sit-to-stand motion transition. Section IV includes the discussion of the experiment results on the soft hardware design for the elderly, as well as the design guidelines for supernumerary robotic limbs for the elderly care. The conclusions and limitations of this paper are enclosed in the final section, which ends the paper.
Ii Robotic Cane as Soft SuperLimbs for Elderly
Ii-a Problem Formulation & Biomechanical Modeling
In order to better adjust the output force and control travel distance of the robotic cane during the sit-to-stand process, it is important to formulate the biomechanical model. The proposal of an active robotic cane with linear motion modifies the original model with an extra “leg”. As shown in Fig. 2
, the human is modeled as a kinematic chain of four-linked rigid mechanism with three degrees of freedom in the sagittal plane. The body segments included are feet, shanks, thighs, and head-arms-trunk as a rigid bar named HAT[Sibella2003Obese]. Anthropocentric values are calculated based on [Leva1996Adjustments]. The sit-to-stand movement of the elderly is slow, and the acceleration is relatively small compared to the weight.
Here are the assumptions for the biomechanical model: 1) The sit-to-stand movement symmetric in frontal and horizontal plane; 2) The feet fix to the ground during the whole movement; 3) Inertial of lower limbs not significant.
The external force is anchored at the body, noted as fixed at point. In the process of natural stand-up, is zero. Point varies with the real situation. Therefore, the case that the subject stands up and stands assisted actively by the robotic cane system ware both described in one model, indicated as Fig. 2.
For the HAT segment,
is the unknown hip moment,
is the mass vector of the HAT,is the vector from the hip to the HAT mass center. is the inertial vector, is the mass inertial moment with respect to the perpendicular from the sagittal plane, both calculated from the sensing data. is the vector pointing from the hip to the location of support force contact point and is the support force on . By Newton’s second law, the force from thigh is calculated.
For the thigh segment,
where is the unknown knee moment, is the mass vector of the thigh, is the vector from the knee to the thigh mass center, is the vector from the knee to the hip. is the unknown reaction force from the shank.
For the shank segment,
where is the unknown ankle moment, is the mass vector of the shank is the vector from the ankle to the shank mass center. is the vector from the ankle to the knee. is the unknown reaction force from the foot.
For the foot segment,
where is the mass vector of the foot, is the vector from the ankle to the foot mass center. is the pressure center, is the total moment from the force plate, , denoted as , is the moment by ground reaction force . is the key component that needs to be compared.
Ii-B System Overview of Robotic Cane as a SuperLimb
The proposed SuperLimb system is illustrated in Fig. 3, where soft and non-intrusive techniques are adopted to accommodate the elderly need in ADLs. A pneumatically-driven linear cylinder is used for lower limb support, which provides a reasonably large and compliant thrust force within a safe operating pressure. The overall shape of the pneumatic cylinder is redesigned as a cane that is commonly used by the elderly for body weight support. Considering the reaction force from the cylinder to the human body during physical support, such force may be too much for the elderly to carry. Therefore, an inflatable vest is added to the system, which can be a regular vest with customized design patterns on the outside, as long as individual air-bag arrays are embedded within to provide distributed support during motion. The depth camera is used to collect depth images that are privacy-safe even at home environment to detect the intention from the cane users during motion transition. Data collected from the depth cameras are then transmitted to the controllers of the robotic cane and inflatable vest, so that the elderly users may not need to worry about the operational control of the valves during motion transition, reducing the stress and complexities in manual control and valve adjustment.
The robotic cane is to be used in the home environment at places such as the bedside, toilet, or the chairs in the living room, as shown in Fig. 1. The assumption is that the elderly user will have the vest put on at first. Then, when moving towards the edge of the seat with the robotic cane in hand, the depth camera will capture the movement and detect the intentions after that. The processed data on user intention will then be transmitted to the on-board controller of the robotic cane to feed the modified biomechanical model with a robotic cane. Prior model tuning is required during the initial installation of the whole system to adjust the individual need of the elderly user. Once a sit-to-stand intention is detected, the inflatable vest will be pressurized, and then the robotic cane will start to move, providing axial thrust to assist the elderly user in completing the sit-to-stand process.
The adoption of “soft” hardware is a crucial design feature of the overall system. The pneumatically-driven cylinder in the robotic cane aims at providing a gradual and compliant, yet powerful, thrust while providing lower limb support. The inflatable vest is used as a soft interface between the cane and the wearer, where a flexible joint will be used to connect the vest with the cane. In this way, the thrust force provided by the cane will be distributed through the vest to act on the wearer to avoid impact or hard push. The depth-camera is a much less intrusive sensing solution, providing more detailed information to detect whole-body user intention. After providing the assistance, the vest will deflate, and the cane will be adjusted to a locked position, so that the elderly user may use the robotic cane usually as a cane for walking assistance.
Ii-C Design of the Robotic Cane
Assistive cane or walking stick is a common type of crutch device used by the elderly for weight redistribution, improved stability, and tactile information about the ground [Kaye2000Mobility]. Ten percent of US adults over 65 years use a cane[Bradley2011Geriatric]. However, there remains a limited design to robotize the cane for elderly usage. A potential drawback for a robotic cane would be an increase in weight and size, which may cause discomfort by the elderly users. Engineering design issues on this problem may be solved by using more advanced material and a light-weight power source. In this paper, we propose a pneumatically-driven robotic cane to explore the potential design issues related to sit-to-stand assistance for the elderly usage.
Before establishing a proof-of-concept prototype, we should first consider requirements especially length, travel distance, and bearing capacity. Mean height and mean weight of elderly men are 173.4 and 88.3, used as references [MeanWeight2018]. The upper limb accounts for approximate 53% of total height [Leva1996Adjustments]. With the assumption that hip height change before and after sit-to-stand is equal to the thigh length, the travel distance is roughly 30% [Leva1996Adjustments]. Consequently, the major design requirements can be summarized as Table I.
We modified a pneumatic cylinder of 32 cross-sectional diameter with 500 travel distance and 0.8 maximum pressure by adding a handle on top with controllers and connectors inside, as shown in Fig. 5(a). The maximum thrust force of a single cylinder is about 644, meaning that a double-cane design can provide a lifting force for an object of about 130, which satisfy the requirement. As our intentional use is for assistive support, a single cane design is sufficiently powerful to help an elderly with reasonably motor functions for sit-to-stand purposes.
A 3D printed T-shape handle is designed and assembled to the air cylinder. The current version is still manually actuated, and a button is placed on top of the handle to operate the valve. The L-shaped air connectors reroute the tubes along the cylinder. The wires from the button together with tubes go through the handle to the back for connection. The electromagnetic valve controlled by the button is to switch the inlet tube. The hook under the handle is useful during attachment to the vest. Since the cane is not rigidly attached to the body, there remains certain flexibility from the cane when providing the thrust force to adjust to the human posture during sit-to-stand.
Ii-D Elderly Wearable using an Inflatable Vest
An inflatable vest is also added to the system to distribute the thrust force provided by the robotic cane evenly. The outer and inner layers of the vest are regular clothes, while the middle layer is air inflatable. As a proof-of-concept, an inflatable swimming jacket is sewed to the inside of the vest. When deflated, the vest is almost the same as a regular vest. Once inflated, the vest will pop up and conform to the shape of the body, forming a tight fit between the body and the vest. Attachment rings are sewed on to the side of the vest under the arm, above the waist line, which are to be connected with the hook on the cane during usage. When the cane is pressurized, the cane will push the rings on the vest, and the thrust force will be evenly distributed through the inflatable swimming jacket inside to achieve compliant support. As a result, the vest is able to hold the human body while the thrust force from the cane increases.
Ii-E Privacy-safe System for Intention Detection
The pattern of human motion can be described by the temporal and spatial movement of their skeleton. The development of the 3D depth camera has made it possible to trace human body skeleton joints economically. Microsoft Kinect for Azure released its body tracing software development kit [shotton2013real-time], by which the temporal-spatial positions of 32 human body skeleton joints are available. For privacy consideration, only the 3D position of each joint is recorded, rather than RGB-video. Firstly, the body skeleton joints are projected on the sagittal plane. The angles of ankle, knee, and hip are then calculated. As shown in Fig. 3, by importing the joint information of the current state, the controller is able to calculate the error using the biomechanical model and adjust the robotic cane output force.
Two experiments are designed to investigate (1) the feasibility of the robotic cane and inflatable vest to provide sit-to-stand assistance; (2) the rationality of the biomechanical model; and (3) the posture recognition during sit-to-stand against motion capture system for intention detection.
Iii-a Experiment Setup
Eight force plates (1200Hz, Bertec) were embedded in each step of a stair. In this experiment, at most three force plates were needed. Therefore, two tables were manufactured for elevation so that we could have a flat plane with three sets of independent force data collected, as shown in Fig 6(a). A depth camera (60Hz, Kinect for Azure) and a twelve-camera motion capture system (120Hz, Motion Capture) recorded the whole process synchronously, as shown in Fig 6 (b), (c). The chair with 40cm in height was used in the experiments.
Iii-B Experiment Procedure
Each subject was required to finish two trials of the experiments, standard control group and experimental group. In the standard control group, the subject was expected to perform sit-to-stand with arms folded over the chest and without any external assistance. While in the experimental group, the subject stood up with robotic cane system assisted. Initially, the subject was seated on the chair on the middle force plate. The feet were placed on the front in parallel while the cane placed on the back one in the cane experiment. Nothing else contributed to the applied forces. In the initial position, the subject’s buttocks should be right at the back of the chair seat. Hands did not push off when starting. The first head and trunk orientations were not controlled. Subjects did not change the feet position once the initial state was established.
Each trail of experiments was repeated six times. For the control group, the rising process should follow the beat of a metronome, 60 beats per minute. The purpose was to restrict two trails of experiments within the similar time frame. Subjects were able to rest if needed. Force plates, the depth camera, and the motion capture system recorded the experiments synchronously.
shows the mean vertical ground reaction force on each trial. More detailed values, mean and standard deviations, of ten ground reaction force parameters are shown in TableII, which characterized the process associated with falling risks of the elderly [Yamada2005Parameters, Yamada2009Relationships]. was the force magnitude at hip lift-off, while was the peak ground reaction force. started from beginning to hip lift-off, started from hip lift-off to peak reaction force and started from peak reaction force to completion of the sit-to-stand movement. , , were impulse during , , , respectively. The last two parameters were velocity, describing the change rate of ground reaction force, between hip lift-off to peak ground reaction force, and during knee-hip extension.
Fig. 8 (a) shows the torque acting on the ankle of real ground reaction force and calculation result from the biomechanical model. Fig. 8 (b) shows the detected joint angles from the depth camera against the motion capture system. The black lines were both ground truth values, while the red lines were calculated or detected results.
Iv-a “Soft” Hardware for the Elderly Care
Iv-A1 The Inflatable Vest as the Wearable
In the proposed system, the soft wearable, inflatable vest is a key component, which builds a bridge between the human and robot. Instead of popular applications as actuation, here we present "support" for the soft robot application. It works to support the operator’s upper body with comfort as well as safety and prevent stress concentration considering narrow contact area like a belt. Besides, a two-step setup is as simple as wear a normal jacket for the elderly, indicated in Fig 4.
However, what we are curious about is that besides supporting, what else can the wearable do? The depth camera observes and detects the sit-to-stand movement from a side view currently. At least three cameras are needed in different locations of a home environment. The wearable gives an alternative for sensor integration. Used every day, the wearable, which is embedded with an inertial measurement unit (IMU) and a camera in the front, may be one of the simple ways for sensing. No other device is required for extra wear. The IMU is capable of body posture detection, while the camera "sees" the environment, considering bedside, chair, and toilet, which is level 2 in Fig. 9, a different design level from the proposed system. Apart from enabling the robot to sense the human intention, the wearable serves as a feedback intermediate between the robot and the human. Disney research put forward a force jacket that used airbags for feel effects in the virtual reality entertainment [Effectofmusclestrength]. Inspired by it, some of the inflatable parts of the wearable can be used as feedback signals. Thus, both forward and backward sensing from the human to the robot are achievable.
Iv-A2 The Pneumatic Cylinder as the Robotic Cane
Fluidically driven, the robotic cane is another critical hardware in the system. The sit-to-stand mechanism assisted by the robotic cane can be analyzed by comparing three sets of ground reaction force data in Fig. 7 and Table II. For the human-robot interaction in the experimental group, there was a noticeable time delay in the motion between the robotic cane and the subject. The robotic cane moved first and gradually transmitted the force to the human body. Thus, the subject took longer both in and than the robotic cane. Before the subject reached the peak value of , the force on cane had started to decrease. Almost when the subject started to perform knee-hip extension, indicated as the valley of red dash line, the force on cane decreased to its weight.
The advantages of using the robotic cane system are fewer efforts for lower limbs and lower fall risk when we compare the force on subject of standard control group and experimental group. Force, and , and velocity absolute values of the operator, and , are good reflection. The experimental group showed smaller values of all the mentioned parameters than the standard control group. Thus, the force curve using the robotic cane was smoother and flatter. More importantly, the overshoot showing in the control group disappeared in the experimental one, which means that the robotic cane successfully compensates the portion of the force larger than the body weight. Thus less muscle strength and fewer efforts are required by the operator shortly after the hip lift-off phase. We can also conclude from of the robotic cane that an external force of 52% of body weight is sufficient for sit-to-stand assistance. Hip lift-off and Knee-hip extension phases are useful in predicting falling risks [Sittostandfromprogressively]. The shorter these phases cost, the easier for the elderly to keep balance when standing up. When standing up assisted by the robotic cane, both and had smaller values than that of the control group. Therefore, the robotic cane shows good performances in supporting the user and reducing potential fall risks.
Iv-A3 The Biomechanical Model with Depth Camera as a Non-Intrusive Sensor
In order to verify the theoretical model, the torques of actual and calculated ground reaction force from the biomechanical model acting on the ankle were compared. From Fig. 8 we can tell that there was a phase shift during the hip lift-off phase and some differences in the torque magnitude due to the assumptions for simplification, which were not exactly true. One possible reason was that the ankle position shook a lot during each cycle and its error could not be neglected compared to its change within centimeter range. Another possible explanation was that the error of each phase was independent from the others. Consequently, to improve the model, further calibration on with standard posture is needed for error reducing.
We used the Kinect Azure SDK to detect the human skeleton information in the sit-to-stand action. The algorithm performed well when the range of the movement was big while the detection precision was not good enough when the range of the movement is small. Since the change in the ankle data was relatively small, within 10 degrees in a cycle, little disturbance and error would lead to the oscillation and detection damages. Many other approaches based on deep learning can offer a robust understanding of the human activities[YeungA]. It is possible to use the biomechanical model to retrieve the human activities from the depth images adjusting the output force of the robotic cane.
Iv-B Design Guidelines for the SuperLimbs
A critical question of the SuperLimbs is the communication between the human and extra limbs. As shown in Fig. 9, different brain-muscle conditions can be categorized into three classes. The corresponding demands for design are shown on the right. Here, the proposed robotic cane system is used as an example. For those whose brain performs well while the muscle undergoes minor degeneration, for instance, the healthy elderly and the climbers, a regular cane with environmental perceiving sensors is sufficient. The SuperLimbs of this level augments human capabilities, [Gonzalez2018Design] is an excellent representative. For those who undergo a significant decline in physical condition while mild in cognitive, for instance, the elderly suffer from motor ability lost and the disabled, additional actuation besides sensors is necessary. The SuperLimbs of this level can help the user to regain normal abilities. For those who suffer from severe brain and muscle disable, a more advanced system with the fusion of multi-modality sensors is significant. The SuperLimbs of this level compensates both cognitive and physical capacities.
Based on different requirements, different levels of brain and muscle function can be remixed in other ways. We focus on the elderly of the second-level while providing an integrated system of the highest level for the entire home assistance in our proposed system.
The soft robot system, which focuses on bed, chair, and toilet as three main places at home, aims at assisting the elderly during the sit-to-stand process. Three components are included in the design, the 1-DOF pneumatically-driven robotic cane, the inflatable vest as the soft human-robot interface, and the non-intrusive depth camera as an ambient sensor. Once the human intention is detected, the system will work automatically, inflating the vest and transmitting force in the stand-up direction, which is under real-time control. The proof-of-concept prototype performed well in compensation of ground reaction force during both the hip-lift-off and knee-hip extension phases according to the experiment. It eliminated the overshoot and results in relatively smooth and flat ground reaction force on feet, which reduces fall risks and requires fewer efforts for the elderly. Finally, we summarize a three-level design guideline, providing a reference to SuperLimbs as the assistive robot design corresponding to the demanding community with different brain-muscle conditions.
However, some issues need to be further addressed and worked on, like power source, which would be either in portable size or fixed on a specific location, friction with the ground. Therefore, the next steps for the soft robot system include calibration and modification on both model and image identification algorithm, implementation of closed-loop control on the pneumatic cylinder, testing with more human users, especially the elderly users, and improvements on design details.