Later year’s increased patient strain on the healthcare system has increased the focus on efficiency improvements within the healthcare system. Increasing healthcare costs as well as a continuous rise in emergency department consultations for inappropriate and non-urgent conditions represent an incitement to treat illnesses at the correct level of care (VSMG14, ). The number of visits to the Swedish emergency departments in Stockholm are increasing by 4.5% annually. Not only the emergency departments are experiencing an increased strain, visits to the out-of-hours clinics and primary care clinics have increased with 6.1% and 3.1% respectively (SLL13, ). This is a phenomenon that is not unique for Sweden, but can be seen in other western countries as well with up to a 40% increase in primary care consultations during the last 20 years (HV09, ; LEME08, ; VSMG14, ).
For many care-seekers, the first line of contact with healthcare services is the Swedish Healthcare Direct (SHD). The SHD is an on-call telephone nursing service similar to the NHS Direct in the UK, LINK in Canada and Health Direct in Australia. Telephone nursing is a common work procedure for nurses in primary care, and is described by Ka09 as performing medical assessments over the telephone, while at the same time providing care with the aim of supporting, strengthening and teaching the callers and guiding care-seekers to the correct level of care. The nursing care provided is based on the caller’s individual needs, and the level of care that is recommended to the callers depends on the nature of and the urgency of their symptoms. A computerized decision support tool aids the nurse in making medical assessments, and provides a shared basis for medical decisions regardless of the nurse’s experience and skills and independent of geographical location, socio-economics or other demographic factors. The system is symptom-based and designed as a checklist where questions relating to the callers symptoms is suggested, and level of urgency is indicated to the nurse (Kam13, ). Through a reduction of the amount of unscheduled visits, the aim of telephone nursing from an economical point of view is to improve planning and making more efficient use of healthcare resources (MSM07, ). But telephone nursing has been found to be highly beneficial for the caller as well in terms of receiving better information, saving time and costs as well as reducing anxiety levels (GG11, ). In addition to the benefits above, telephone nursing has presented many nurses with new and exciting opportunities within healthcare services and many of them enjoy their work and feel that they are offering a worthwhile service (KOMM02, ).
Telephone nursing has been found to be a medically secure and cost-effective form of care, with recommendations assessed as medically adequate in 97.6% of the cases (MSM07, ) and with compliance rates ranging from 81.3% (MSM07, ) to 91% (KCH10, ). Between 30-50% of the calls result in self-care advice (KCH10, ; MSM07, ; GUMA16, ). Previous research has also found that care-seekers are highly satisfied with this service. However, care-seekers that receive a recommendation to practice self-care are less satisfied than those referred to medical care, indicating that their needs of care are not fully met (GUMA16, ).
Modeling the effects of telephone nursing is an important tool in evaluating telephone nursing’s effect on healthcare utilization, as well as enabling the evaluation of possible changes in telephone nursing services. A valid model that describes the effect of the nurse’s recommendation in relation to patients’ intended actions prior to consultation allows us to understand and evaluate different effects in detail. It also enables us to make predictions about reductions in healthcare costs. Earlier research (GUMA16, ) has found that telephone nursing does not always meet care-seekers needs of care. This underlines the importance of continuously working to improve services so that they answer to the needs of the care-seekers.
The aim of the study was to model the effects of telephone nursing. First, we wanted to find the model that best describes the effects of telephone nursing. From this model we then wanted to make inference about patient and the effects of telephone nursing with regards to healthcare utilization and potential cost savings.
2 Materials and methods
In this study a model was developed and validated based on data collected in a cross-sectional study that aimed at exploring the influence of self-care advice on patient satisfaction GUMA16 . The setting of the study was the Swedish Healthcare Direct (SHD) in Northern Sweden. Northern Sweden is a large but sparsely populated area with long distances between inhabitants, hospitals and service providers. Data were collected using a questionnaire that was a further development from an existing evaluation of patient satisfaction and healthcare utilization among callers to the SHD, and details regarding the construction and content of the questionnaire has earlier been published in GUMA16 .
With minor changes to SHD procedures, similar data could be retrieved directly from the SHD logs (in the computerized decision support tool) in combination with patients’ medical records. This means that the same model is applicable on a much larger scale, providing a national evaluation tool of the effect of SHD on healthcare utilization.
The items in the questionnaire that generated data to the model development was ”What was your intended action prior to telephone nursing consultation?”; ”What action did you undertake after telephone nursing consultation?” and ”What was the nurse’s recommendation?”. Four increasing levels of care were proposed and coded as follows: 0=self-care; 1=primary care clinic; 2=out-of-hours clinic and 3=emergency department.
Eligibility criteria was all callers to the SHD in Northern Sweden during one week in the winter of 2014. A sample size calculation in GUMA16 indicated that the dispatch of 500 questionnaires would generate sufficient data. The questionnaire was dispatched to 500 randomly selected callers to the SHD, and study participants were selected using randomization lists from the SHD registers. All questionnaires were dispatched within 8 days of the call to the SHD, and were returned within 28 days of dispatch. If the respondents had made more than one call to the SHD during this week they were asked to reply regarding their last call to minimize the risk of recollection bias. Five questionnaires were returned unopened because of wrong address, and two study participants had deceased after the call. In total 225 persons returned a completed questionnaire, giving a response rate of 45.6%. The study sample consisted of 69.3% women and 30.7% men, reflecting well the proportions of callers to the SHD. The mean age in the sample was 48.15 years, ranging from 17–93 years. The majority of respondents were born in Sweden (93.3%) and cohabiting (79.9%), and the study sample displayed a good representation of the general population in Northern Sweden (Table 1). The number of callers referred to the different levels of care was 75 (33.3%) to self-care, 64 (28.4%) to primary care; 40 (17.8%) to out-of-hour clinic and 53 (23.6%) to the emergency department. The complete dataset used in this study, i.e. number of individuals in each category and their corresponding explanatory variables, is retrievable from Fig. 2.
|Highest completed education level|
|Compulsory school||42 (19.0)||(12.0)|
|Upper secondary school||85 (38.5)||(52.0)|
|Tertiary education||94 (42.5)||(36.0)|
Source: SCB14 .
The outcomes in this study are patient behavior, effects on health care utilization and potential cost savings. The predicted variable is patient’s action undertaken after receiving telephone nursing and the explanatory variables are intended actions prior to consultation and the nurse’s recommendation.
Even though the number of explanatory variables are limited, standard ordered-logit and ordered-probit regression modelsKru14 ; GCSR04 result in invalid descriptions of the observations in our study and fail the model validation step (described in Section 3.1.1). If a model is not able to describe the observed data, then inference based on the model is questionable. Instead, a more flexible Bayesian ordinal regression model is proposed to describe the effects of telephone nursing on the level of care finally chosen by the caller after the consultation.
The main reasons for applying Bayesian analysis are: (i) to easily generalize standard models of ordinal data Kru14 ; GCSR04 ; GH07 (ii) to avoid non-identifiability problems in ordinal regression (e.g. separation and collinearity) by incorporating weakly informative priors proposed by GJPS08 ; (iii) to provide richer inference and avoid the common problems (e.g. dependencies on the sampling and testing intentions) associated with p-values (Kru13, )
; (iv) to make direct probability statements about the effect given the observed evidence(CSAT03, )
(e.g. the studied effect given the observed data is inside the confidence interval with a measurable probability).
The Bayesian ordinal regression model is a generalization of the more commonly used ordered-logit and ordered-probit regression models and adds the necessary flexibility to obtain a valid description of the observed data in this study. The considered model follows Kru14
for ordinal regression and include the following two necessary extensions to the standard logit and probit models: heterogeneous standard deviation, and additional robustness against outliers to account for possible contamination described next. These extensions are also suggested inKru14 for robust logistic- and ordinal regression.
Data were analyzed using the statistical packages for scientific computing with Python (Oli07, ; MA11, ). The inference is based on Markov Chain Monte Carlo methods (GCSR04, ; Kru14, ; CSI00, ) and calculated from 100,000 samples from the posterior distribution (after burn-in) with converged chains. The confidence intervals are here based on the highest posterior density interval Kru14 .
The data and the results from the ordinal regression can be seen in Fig. 1. The horizontal axis represents the intended action prior to consultation () and the vertical axis represents the nurse’s recommendation (), both categorized into the four levels of care marked by the dotted grid. The levels are defined as 0=self-care; 1=primary care clinic; 2=out-of-hours clinic and 3=emergency department. The scatter plot with color-coded numbers (0–3) shows the level of care finally chosen by each caller after receiving telephone nursing (), for the same levels of care as the explanatory variables ( and ). For visualization purposes, the numbers are randomly scattered around their corresponding categorical integer coordinates marked by the dotted grid.
3.1 The model
The model can be summarized by the dependent variable , which is the th patient’s action undertaken after receiving telephone nursing, and its two explanatory (or input) variables: the th patient’s intended action prior to receiving telephone nursing; the nurse’s recommendation given to the th patient. The data (i.e. , for patients and explanatory variables) is categorized into the four increasing levels of care defined earlier. We summarize the model in three parts below. The first part is devoted to the regression model, while the second and third parts deals with the proposed extensions to the standard ordered-logit and ordered-probit models.
Part I: The mean utilized care (on the underlying metric scale) is given by a second order polynomial
and is the result after the model order selection procedure, described in Section 3.1.3.
The standard ordered-logit or ordered-probit alternative assumes that the independent errors follow either a logistic or a normal distribution, respectively,Kru14 ; GCSR04 . However, neither distributions are flexible enough to describe the observed data in this study. Two necessary extensions to the description of the independent errors are proposed here that when combined lead to a valid description of the observations:
Part II: A single distribution for the error term is replaced with a mixture model of two distributions with different scales to account for outliers, following the recommendations in Kru14 for robust logistic- and ordinal regression. Other alternatives to increase the robustness to extreme values are available (see e.g. GCSR04, ; Kru14, ) but we choose the mixture model for simplicity and also for clarity as it gives a measure of the departure from the standard ordered-probit model.
In addition to providing a valid description of the observations, these extensions also contribute to interesting inferential results regarding: patient behaviour, effects of opening hours, variations in predictability, effects of perceived risks, proportion of extreme answers etc., that would otherwise not be inferred.
We return to these tree parts in the subsequent sections below. A detailed model definition with the proposed extensions to the standard ordinal models is available in A.
3.1.1 Model validation
Model validation is a central part in justifying the model and more importantly to justify inference based on it (see e.g. GCSR04, ; GH07, ). We try to detect flaws in the model’s predictive ability following the techniques recommended in GMS96 ; GCSR04 . In Fig. 2, histograms based on the model’s predictions are seen together with the actual data represented by the black cross. The histogram bars represents the model’s predicted level of healthcare after consultation and the black box shows the 95% prediction interval. Based on the predictions (see B), there is little evidence that would indicate significant discrepancies between the proposed model and the actual observations (GCSR04, ).
3.1.2 Error rate
The error rate, defined as the proportion of cases for which the prediction of the final action is wrong (GH07, ), is 23.6% using the proposed model. This means that the model correctly predicts 76.4% of the patients’ action undertaken after consultation, given the intended action prior to consultation and the nurse’s recommendation. This can be compared to a prediction of 25% by chance and 43.1% using a model without the explanatory variables.
A complete probability description of all care levels was shown in Fig. 2 and revealed no significant discrepancies between the model predictions and the actual observations.
3.1.3 Model selection
The model selection step is organized in the same three parts as was introduced earlier in Section 3:
Part I: The first part is to find the appropriate combination of the explanatory variables to describe the expected level of care in (1). All model structures up to the full quadratic form (i.e. second order polynomial including interactions) are evaluated and the model structure proposed in (1) returns the overall lowest Watanabe-Akaike Information Criteria (WAIC) Wat10 ; GHV14 . The use of second order terms is also motivated by examining the clustering of the data in Fig. 1 with respect to the two explanatory variables, indicating non-linear behavior for large values of and low values of .
Part II: The second part is to find the appropriate distribution to use for the error term to cope with outliers. We included outliers in the statistical model using a mixture model with a contamination parameter adjusting the probability mass in the tails of the distribution, see A for more details. This is the first extension to the standard ordered-logit or ordered-probit model explained previously, and is needed to pass the validation steps described in Section 3.1.1. Including the contamination parameter also decreases both the error rate (Section 3.1.2) and the WAIC, indicating the need of robustness against extreme answers.
Part III: The third part is to find the appropriate combination of the explanatory variables to describe the uncertainties in the level of care ( in A). The difference in uncertainty visible in the data in Fig. 1 at different values of the explanatory variables supports the use of a heterogeneous standard deviation model. The best predictive performance and lowest WAIC was given by using four separate standard deviation parameters, i.e. with a separate parameter for each class of intended action prior to consultation . This choice reflects the differences in uncertainties and patients’ risk reducing behaviors found in Section 3.2.
3.2 Inference from the model
This section starts by summarizing the model, followed by two additional subsections on healthcare utilization and cost savings.
Part I: The regression parameters in (1) describes the effect of each explanatory variable on the utilized level of care .
The estimated effects given the observed data are summarized in Table 2. We see a linear positive effect () of intended action prior to consultation () on the utilized level of care. The effect of the nurse’s recommendation () is positive and quadratic and when the nurse recommend a high level of care it is the dominating in (1) and explain the top row of all trees at in Fig. 1. From Table 2 we conclude that:
If the intended action prior to consultation increases one care level, then the mean level of utilized healthcare increases 0.35 (on the underlying metric scale).
If the nurse’s recommendation increases:
from self-care to primary-care, then the mean level of utilized healthcare increases 0.52 (on the underlying metric scale).
from primary-care to out-of-hours clinic, then the mean level of utilized healthcare increases 1.24 (on the underlying metric scale).
from out-of-hours clinic to emergency department, then the mean level of utilized healthcare increases 1.96 (on the underlying metric scale).
The relative effects on health care utilization at different combinations of and is given in Section 3.2.1. The estimated value of is credibly zero but included as it reduces the WAIC. Also, if is significant we should include lower order terms (Kru14, ; GH07, ).
The parameter values generate the threshold curves shown in Fig. 1 and consequently also the outcome probabilities for each level of care given the values of the corresponding explanatory variables. The outcome probabilities are illustrated as histograms in Fig. 2.
Part II: Estimates of the contamination parameter are shown in Table 2. As zero is outside the interval we conclude presence of extreme values in the questionnaire responses that departs from a single normal distribution. This is an additional evidence of the necessity to extend the standard ordered-logit and ordered-probit models with a mode flexible distribution of the error term.
Part III: The standard deviation parameters at different levels of intended action, shown in Table 2, depends on the caller’s intended care level prior to consultation.
The value of is high. It seems plausible that the final action is more uncertain when the patient’s intended action is the highest level of care (3=emergency department) and the recommendation is otherwise.
The value of in Table 2 is low, indicating that opening hours affect both the intended action and the recommended action. As seen in the data in Fig. 1, if the intended action is 2=out-of-hours clinic, it is likely that the telephone call occurred during these hours. Consequently, the intended, the recommended and the final action are more restricted with most outcomes at . This prevents the spread in outcomes over all levels of care, resulting in a lower standard deviation.
|The effects of the explanatory variables|
|(-0.76, -0.05)||(0.17, 0.54)||(-0.33, 0.67)||(0.15, 0.58)|
|The uncertainties at different patient’s intentions|
|(0.43, 1.02)||(0.46, 1.15)||(0.18, 0.40)||(1.16, 4.19)|
|The contamination parameter|
3.2.1 Effects on healthcare utilization
The slope of the threshold curves in Fig. 1 reveal the relative effect of the two explanatory variables and can be interpreted as follows:
If the curves are vertical, the nurse’s recommendation () has no effect on the final action undertaken, and the patient is guided by their intended action () prior to consultation only.
If the curves are horizontal, the patient’s intended action prior to consultation () has no effect, and the patient is guided solely by the nurse’s recommendation ().
If the curves lean with a derivative of negative one, i.e. from the upper left corners to the lower right corners in the grid, the effects of the intended action and recommended action are the same.
Overall, the threshold curves in Fig. 1 lean with a slope that is larger than negative one in Table 3 and we conclude that the nurse’s recommendation is the dominating effect in the majority of the cases. The exception is when the intended action prior to calling is the emergency department (), indicating that the recommendations has little or no effect compared to the intended action. A plausible explanation is that the perceived risk to follow a self-care advice () is higher, and the patient is then guided by the intended action.
From the reciprocal slope we conclude that if the recommendation is the emergency department () the effect is on average 7.09 times higher than the effect of the intended action prior to consultation. This can be compared to a recommendation of self-care () which gives a factor of 0.58 and credibly zero according to the 95% interval. The relative effect increases with the nurse’s recommended care-level and a plausible explanation is that disregarding a higher-level recommendation implies a higher risk compared to disregarding lower-level recommendation.
Expressions of the derivatives are available in C.
|(-0.26, -0.09)||(-0.28, -0.09)||(-0.30, -0.09)||(-0.34, -0.09)|
|(-0.33, -0.12)||(-0.38, -0.13)||(-0.46, -0.13)||(-, -0.18)|
|(-0.43, -0.18)||(-0.57, -0.18)||(-, -0.28)||(-, -0.36)|
|(-2.32, 1.18)||(-4.66, -1.43)||(-7.89, -2.56)||(-11.57, -3.48)|
3.2.2 Potential cost savings
The estimated cost savings in the analysis are based only on the direct costs associated with the first action undertaken by the patient after consultation and do not include additional savings as a result of improved planning of healthcare resources, reduction of unscheduled visits and guidance to the correct level of care. The results presented here are based on flat-rates from the county council where the telephone nursing consultation costs 96 SEK, a visit to the primary care clinic is approximately 2000 SEK, a visit to the out-of-hours clinic approximately 3000 SEK and a visit to the emergency department is approximately 4500 SEK.
The predicted cost (in SEK per patient) that corresponds to the utilized healthcare after consultation can be seen in Fig. 3 (in red). For comparison, Fig. 3 also show both the predicted cost if all patients would follow the nurse’s recommendation (in blue) and the predicted cost if the patients would do as they intended prior to consultation (in black).
The ratio of the predicted cost associated with the final action undertaken after consultation and the intended action prior to consultation is 0.97 (0.87, 1.06) per patient. The expected cost savings associated with telephone nursing is 3.3% compared to the intended action prior to consultation. If the patients would follow the nurse’s recommendations in all cases the predicted cost ratio is 0.87 (0.71, 1.02) per patient, corresponding to an expected cost saving of 12.7% of all patients.
Good statistical models that describe the effect of telephone nursing on healthcare utilization are essential in order to study its impact on healthcare resources and evaluate effects of future changes in telephone nursing procedures. The aim of this study was to model the effects of telephone nursing on patient behavior, healthcare utilization and infer potential cost savings. In this study we present a Bayesian ordinal regression model that captures the complex relationship between the nurse’s recommendation, the patients’ intended action prior to consultation and the patients’ final action undertaken after receiving telephone nursing. We found no evidence to suggest any model deficiencies based on posterior predictive checks on standard discrepancy measures.
The main finding from this study is a model that predicts 76.4% of the patients’ action undertaken after consultation. This number is based only on the most probable category predicted by the model, i.e. the model’s “best guess”, but it in addition to this it was shown that the model gives a valid probability description of all care levels after consultation.
Inference reveal that the effect of the nurse’s recommendation is lower when the patient’s intended action prior to consultation is the highest level of care. If the recommendation is the emergency department the effect of the recommendation is on average a factor of 7.09 times higher than the effect of the intended action prior to consultation. This can be compared to a recommendation of self-care which gives a factor of 0.58 and credibly zero according to the 95% interval.
We found that the final action was more uncertain when the patient’s intended action is the highest level of care and the nurse’s recommendation is otherwise. It is even plausible that the recommendation by the nurse has no effect on the final outcome when the patient’s intended action is a visit to the emergency department. The results reveal a risk reducing tendency among the callers. Consistent with several theories of health behavior, a high perceived risk of harm will according to BWC04 encourage people to take action to reduce their risk. Perceptions of risk has been found to depend on several parameters such as age, sex and health status, and that some personality types overestimate the probability of disease and/or underestimate the effects of preventive actions such as self-care (EJ13, ). This underlines the importance of administrating structured nursing with the specific aim of providing care-seekers with reassurance, as feeling reassured might reduce perceptions of risk. Reassurance has been found to be highly influential on patient satisfaction (GUMA16, ), and persons satisfied with the nurse interaction has been found to be nearly four times more likely to report engaging in self-care activities (WWMJ12, ).
According to Varley201612 a clinically significant reduction in workload could be achieved if all nurses reached a proportion of patients managed by self-care of at least 65%, assuming that referral to self-care was appropriate. A result in our study was that cost savings based only on the patient’s first action undertaken after consultation is estimated to be 3.3% but could be up to almost four times higher if compliance to the nurse’s recommendations was higher. It is plausible that compliance would increase if callers were satisfied and reassured to a higher extent. The cost analysis is far from comprehensive as it neglects other significant cost reduction aspects with telephone nursing, but shows the importance of further investments to strengthen the development of the nurse-client interaction.
In conclusion, the possibility to increase cost savings is a strong incentive to conduct further research aiming at improving persons trust in and tendency to follow the nurse’s recommendations. It is likely that compliance would increase if care-seekers feel reassured and confident that following the recommendations will generate a successful outcome. This implies that the interpersonal relationship that the caller and nurse establish is highly important. To improve telephone nursing compliance, care-seekers need to feel confident that following the recommendations does not involve increased risks. More work is needed in order to further improve nurses ability to meet the needs of care-seekers.
4.1 Study limitations
The data on healthcare utilization was in this study collected through self-reporting. Social desirability bias has been found to result in higher levels of self-reported compliance to self-care advice compared to medical records (OCO02, ). This could imply a possible limitation as no comparison to medical records was performed. For reasons of confidentiality the researchers were not granted access to the medical records of the patients. To reduce social desirability bias, the questionnaire item regarding what action was taken was asked in a neutral manner, and this item was placed before the one asking what advice they had received from the nurse in the questionnaire.
No potential confounders were identified in our material, but it is plausible that psychological factors e.g. concern or anxiety has influenced intended actions prior to consultation. If the need of reassurance has not been adequately addressed through telephone nursing, this might have influenced the action undertaken after consultation. Psychological factors have not been analyzed due to the lack of this data in our material.
The estimated cost savings are restricted to the direct costs associated with the first action undertaken by the patient after consultation and do not include additional savings as a result of improved planning of healthcare resources, reduction of unscheduled visits and guidance to the correct level of care. For instance, even if a patient intended to practice self-care but was recommended a visit to the emergency department, a timely guidance to the correct level-of-care may result in a reduction of the total costs in the long run. To obtain a comprehensive analysis of the savings associated with SHD, aspects such as these must be included in the analysis.
The reported inference (e.g. the predictability, the specific effects of the explanatory variables, cost savings, etc.) are limited to the specific study setting and the telephone nursing procedure given by the SHD. Larger and more rigorous studies are needed to obtain evidence of effectiveness of telephone nursing nationally EKELAND20121
. However, the proposed Bayesian ordinal regression model, the model selection and validation procedure, and the interpretation of the model parameters, threshold curves, derivatives and estimands are not limited to the study setting. The reported procedures are applicable in evaluating, analyzing and understanding the effects of on-call services on healthcare utilization and patient behaviors. The number of care levels can easily be adjusted and in the simplest setting of two levels the ordinal regression model is reduced to a logistic regression(Kru14, ).
With minor changes to SHD procedures, similar data could be retrieved directly from SHD logs in combination with patients’ medical records. This means that the proposed model is applicable on a much larger scale, providing not only a national evaluation tool of the effect of SHD on healthcare utilization but also more inferential evidence in combination with additional explanatory variables that may influence outcome probabilities.
4.2 Clinical implications
What these findings mean in a clinical setting is that telephone nursing has a constricting effect on healthcare utilization, and that guidance to the correct level of care implies a potential to save costs. However, the compliance to nurse’s recommendation is closely tied to perceptions of risk, and this emphasizes the importance to address caller’s needs of reassurance. A trustful relationship between the caller and the nurse carries the potential to reassure and strengthen confidence in the nurse’s recommendation. But for this to happen the organization of telephone nursing needs to allow the nurse to invest time and energy to establish a trustful relationship. It also implies that nurses must acknowledge that the need for nursing care might persist even when the need for medical care is absent or resolved.
The first author, conceived, planned, and executed the data analysis and wrote the manuscript. The second author planned and executed the data collection and contributed to writing the manuscript. Both authors have read and accepted the manuscript, and there are no conflicts of interest.
Conflicts of interest
The regional ethical review board approved this study (dnr 2010-225-31)
What was already known on the topic:
There is an increase in utilization of healthcare resources that cannot be described by population growth or decreasing health.
There are indications that demand and expectations of welfare services increase with increasing gross domestic product and household income.
On-call telephone nursing services is the first line of contact for many care-seekers and aims at optimizing the healthcare system by providing medical assessments, support and guidance to the correct level-of-care over the phone.
What knowledge this study adds:
A valid model that describes the effect of telephone nursing in relation to patients’ intended actions prior to consultation.
The model is an important tool in evaluating telephone nursing’s effect on healthcare utilization as well as to evaluate the effects of future changes in telephone nursing services. Inference allows us to estimate different effects in detail, understand patient behavior and patient predictability, and to estimate potential cost savings per patient.
Telephone nursing has a constricting effect on healthcare utilization but the compliance to nurse’s recommendation is likely to be influenced by the patients’ perceptions of risk. This emphasizes the importance of meeting caller’s needs of reassurance.
The authors gratefully acknowledge all study participants for taking part in the study and Robert Lundqvist at the county council of Norrbotten for the help received with this study.
- (1) A. Vedovetto, N. Soriani, E. Merlo, D. Gregori, The burden of inappropriate emergency department pediatric visits: Why italy needs an urgent reform, Health Services Research 49 (4) (2014) 1290–1305.
- (2) Stockholms Läns Landsting, Genomlysning av stockholms fem stora akutmottagningar, Tech. rep., Hälso- och sjukvårdsförvaltningen (2013).
- (3) J. Hippisley-Cox, Y. Vinogradova, Final report to nhs information centre and department of health – trends in consultation rates in general practice 1995/1996 to 2008/9: analysis of the qresearch database., Tech. rep., The NHS information Centre for Health and Social Care, http://www.hscic.gov.uk/catalogue/PUB01077/tren-cons-rate-gene-prac-95-09-95-09-rep.pdf (2009).
- (4) F. Lega, A. Mengoni, Why non-urgent patients choose emergency over primary care services? empirical evidence and managerial implications, Health Policy 88 (2–3) (2008) 326–338. doi:10.1016/j.healthpol.2008.04.005.
- (5) E. Kaminsky, U. Roseqvist, I. Holmström, Telenurses’ understanding of work: detective or educator?, Journal of Advanced Nursing 65 (2) (2009) 382–390. doi:10.1111/j.1365-2648.2008.04877.x.
- (6) E. Kaminsky, Telephone nursing: Stakeholder views and understandings from a paediatric and a gender perspective, Ph.D. thesis, Uppsala University (2013).
- (7) B. Marklund, M. Ström, J. Mansson, L. L. Borgquist, A. Baigi, B. Fridlund, Computer-supported telephone nurse triage: an evaluation of medical quality and costs, Journal of Nursing Management 15 (2) (2007) 180–187.
- (8) C. Grandchamp, L. Gardiol, Does a mandatory telemedicine call prior to visiting a physician reduce consts or simply attract good risks?, Health Economics 20 (10) (2011) 1257–1267. doi:10.1002/hec.1668.
- (9) E. Knowles, A. O’Cathain, J. Morrell, J. Munro, J. Nicholl, NHS Direct and nurses—opportunity or monotony?, International Journal of Nursing Studies 39 (8) (2002) 857–866.
- (10) E. Kaminsky, M. Carlsson, A. T. Höglund, I. Holmström, Paediatric health calls to swedish telenurses: a descriptive study of content and outcome, Journal of Telemedicine and Telecare 16 (8) (2010) 454–457.
- (11) S. Gustafsson, J. Martinsson, S. Sävenstedt, B. Wälivaara, I. Vikman, Influence of self-care advice on patient satisfaction and health care utilization, Journal of advanced nursing Epub.
- (12) Statistics Sweden, Sweden in numbers, http://www.scb.se/en_/, accessed: 2015-06-02 (2014).
- (13) J. Kruschke, Doing Bayesian Data Analysis: A Tutorial Introduction with R, JAGS and Stan, 2nd Edition, Elsevier Science, 2014.
- (14) A. Gelman, J. B. Carlin, H. S. Stern, D. B. Rubin, Bayesian Data Analysis, 2nd Edition, Chapman & Hall/CRC, 2004.
- (15) A. Gelman, J. Hill, Data analysis using regression and multilevel/hierarchical models, Vol. Analytical methods for social research, Cambridge University Press, New York, 2007.
- (16) A. Gelman, A. Jakulin, M. G. Pittau, Y. Su, A weakly informative default prior distribution for logistic and other regression models, Ann. Appl. Stat. 2 (4) (2008) 1360–1383.
- (17) J. Kruschke, Bayesian estimation supersedes the t test, Journal of Experimental Psychology: General 142 (2) (2013) 573–603.
- (18) N. J. Cooper, A. J. Sutton, K. R. Abrams, D. Turner, A. Wailoo, Comprehensive decision analytical modelling in economic evaluation: a bayesian approach, Health Economics 13 (3) (2003) 203–226. doi:10.1002/hec.804.
- (19) T. E. Oliphant, Python for scientific computing, Computing in Science Engineering 9 (3) (2007) 10–20. doi:10.1109/MCSE.2007.58.
- (20) K. J. Millman, M. Aivazis, Python for scientists and engineers, Computing in Science Engineering 13 (2) (2011) 9–12. doi:10.1109/MCSE.2011.36.
- (21) M.-H. Chen, Q.-M. Shao, J. G. Ibrahim, Monte Carlo Methods in Bayesian Computation (Springer Series in Statistics), Springer, 2000.
A. Gelman, X.-L. Meng, H. Stern,
Posterior predictive assessment
of model fitness via realized discrepancies, Statistica Sinica 6 (4) (1996)
S. Watanabe, Asymptotic equivalence of bayes cross validation and widely applicable information criterion in singular learning theory, Journal of Machine Learning Research (2010) 3571–3594.
- (24) A. Gelman, J. Hwang, A. Vehtari, Understanding predictive information criteria for bayesian models, Statistics and Computing 24 (6) (2014) 997–1016.
- (25) N. T. Brewer, N. D. Weinstein, C. L. Cuite, J. E. Herrington, Risk perceptions and their relation to risk behavior, Annals of Behavioral medicine 27 (2) (2004) 125–130.
- (26) J. Etner, M. Jeleva, Risk perception and diagnostic tests, Health Economics 22 (2) (2013) 144–156. doi:10.1002/hec.1822.
- (27) B. Williams, S. Warren, R. McKim, W. Janzen, Caller self-care decisions following teletriage advice, Journal of Clinical Nursing 21 (7–8) (2012) 1041–1050.
- (28) A. Varley, F. C. Warren, S. H. Richards, R. Calitri, K. Chaplin, E. Fletcher, T. A. Holt, V. Lattimer, J. Murdoch, D. A. Richards, J. Campbell, The effect of nurses’ preparedness and nurse practitioner status on triage call management in primary care: A secondary analysis of cross-sectional data from the ESTEEM trial, International Journal of Nursing Studies 58 (2016) 12 – 20.
- (29) J. O’Connell, W. Towles, M. Yin, C. Malakar, Patient decision making: use of and adherence to telephone-based nurse triage recommendations, Medical Decision Making 22 (4) (2002) 309–317.
A. G. Ekeland, A. Bowes, S. Flottorp,
for assessing telemedicine: A systematic review of reviews, International
Journal of Medical Informatics 81 (1) (2012) 1 – 11.
- (31) A. Gelman, Scaling regression inputs by dividing by two standard deviations, Statist. Med. 27 (15) (2008) 2865–2873.
- (32) A. Gelman, Two simple examples for understanding posterior p-values whose distributions are far from uniform, Electron. J. Statist. 7 (2013) 2595–2602. doi:10.1214/13-EJS854.
Appendix A Model definition
Starting from the expression of the expected level of care given in (1), a latent-data formulation (GCSR04, ) of the model is given by the unobserved or latent outcome with independent errors . The errors are distributed as a mixture of two normal distributions with different scales to account for the presence of outliers. The contamination parameter is estimated from the observed data (along with the other model parameters) to adjust the tail width in presence of outliers (Kru14, ). The observed discrete outcome
is categorized into the four increasing levels of care depending on the threshold values for .
Four different standard deviation parameters, one for each category of intended action prior to consultation, are included in the model. This is motivated by examining the deviation in the data at different levels and it also increases the model’s predictive performance and decreases the Watanabe-Akaike Information Criteria (WAIC) (Wat10, ; GHV14, ) compared to a homogeneous standard deviation model (Kru14, ). The index in returns an integer value of the category (0–3) and distinguishes the four standard deviation parameters with respect to the intended action prior to consultation (). We return to this particular choice in Sections 3.1.3 and 3.2.
The explanatory variables are mean centered and divided by the their standard deviation for computational stability (Kru14, ). To avoid problems with separation and collinearity we incorporate the weakly informative Cauchy prior distributions of the model parameters on the scaled explanatory variables proposed in Gel08 . The prior distribution for the thresholds are normally distributed , where the two extreme thresholds are fixed to meaningful values on the level of care, and (Kru14, ).
For robust inference, the prior distribution for the contamination parameter is beta distributed, recommended in Kru14 for robust logistic- and ordinal regression. This prior distribution has a mean value of
and gives a very low but non-zero prior probability of. The prior distribution for the standard deviation parameters are uniform on a very large scale.
Appendix B Model validation
The model generates 100,000 simulated datasets from the posterior predictive distribution and in each dataset there are 225 new individuals given the same explanatory variables as in the observed dataset. The proposed model seems to generate predicted data similar to the observed data and the 95% confidence interval boxes in Fig. 2 contains 63 out of the 64 crosses and 98.4 (91.7, 99.6)% of the observations in this study. This percentage (and corresponding interval) is matches the predefined risk of 5%, considering multiple comparisons (see e.g. (GH07, )).
To further test the validity of the model we use the number of patients in each care-level as a discrepancy measure (Gel13, ). Based on the replicated datasets, the posterior predictive double-tailed p-vales (Gel13, ) for the number of patients in each of the four care-levels (0=self-care, 1=primary care clinic, 2=out-of-hours clinic and 3=emergency department) are 0.412, 0.923, 0.697 and 0.185, respectively. Considering the mean value and standard deviation of the overall outcomes as discrepancy measures, return p-values of 0.195 and 0.338, respectively.
Appendix C Derivatives of threshold curves
The relative effect of each explanatory variable can be investigated more precisely by examining the derivative of (3) with respect to each explanatory variable
and each threshold . The resulting derivatives are the reciprocal slope in (4) and the slope in (5) of the threshold curves in (3) and shown in Fig. 1. The point and interval estimates presented in Table 3 are easily obtained from posterior samples of the corresponding regression and threshold parameters in (4–4). Note that slope (4) is independent on and is the same for all curves at the specific value, this is also visible in Fig. 1. The reciprocal slope, however, depends on and is different for each curve and value.