Outlier Resilient Collaborative Web Service QoS Prediction

06/01/2020 ∙ by Fanghua Ye, et al. ∙ Huazhong University of Science u0026 Technology UCL SUN YAT-SEN UNIVERSITY 0

Nowadays, more and more Web services are provided by different enterprises and organizations. The proliferation of Web services makes it difficult for users to select the most appropriate Web services among numerous functionally identical or similar service candidates. Quality-of-Service (QoS) is widely employed for describing the non-functional characteristics of Web services, and it has become the key differentiator for service selection. However, users cannot invoke all Web services to obtain the corresponding QoS values due to high time cost and huge resource overhead. Thus, it is essential to predict unknown QoS values. Although various QoS prediction methods have been proposed, few of them have taken outliers into consideration, which may dramatically degrade the prediction performance. To overcome this limitation, we propose a robust collaborative QoS prediction method in this paper. Our method utilizes Cauchy loss to measure the discrepancy between the observed QoS values and the predicted ones. Owing to the robustness of Cauchy loss, our method is resilient to outliers. We further extend our method to provide time-aware QoS prediction results by taking the temporal information into consideration. Finally, we conduct extensive experiments on both static and dynamic datasets. The results demonstrate that our method is able to achieve better performance than state-of-the-art baseline methods.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1. Introduction

Web service is a standard technology for providing interoperability between disparate software applications and plays the key role in service-oriented computing (Bouguettaya et al., 2017). Over the past few years, a large number of Web services have been published as indicated by the Web service repository–ProgrammableWeb111https://www.programmableweb.com/. The proliferation of Web services brings great benefits in building versatile service-oriented applications and systems.

It is apparent that the quality of service-oriented applications and systems relies heavily on the quality of their component Web services. Thus, investigating the quality of Web services is an important task to ensure the reliability of the ultimate applications and the entire systems. The quality of Web services can be characterized by their functional and non-functional attributes. Quality-of-Service (QoS) represents the non-functional aspect of Web services, such as response time, throughput rate and failure probability

(Papazoglou et al., 2007; Wu et al., 2015). Since there are many functionally equivalent or similar services offered on the Web, investigating non-functional QoS properties becomes the major concern for service selection (El Haddad et al., 2010; Zhang et al., 2011). However, the QoS value observed by users depends heavily on the Web service invocation context. Hence, the quality of the same Web service experienced by different users may be relatively different (Shao et al., 2007). For this reason, it is important to acquire personalized QoS values for different users. Considering that users cannot invoke all Web services to obtain personalized QoS values on their own due to high time cost and huge resource overhead (Wu et al., 2015; Zhang et al., 2019), predicting missing QoS values based on existing observations plays an essential role in obtaining approximate personalized QoS values.

For QoS prediction, collaborative filtering is one of the most widely adopted methods (Wu et al., 2015). Existing collaborative filtering methods can be grouped into memory-based methods and model-based methods. The memory-based methods aim at utilizing similar users (or similar Web services) for a target user (or a target Web service) to predict unknown QoS values (Zou et al., 2018; Chen et al., 2020). Although memory-based methods are straightforward and easy to implement, they usually suffer from the data sparsity problem (Zhang et al., 2019; Zheng and Lyu, 2013). The model-based methods aim at building global prediction models based on existing QoS observations to predict unknown QoS values (Zhang et al., 2019). Among model-based methods, matrix factorization (MF) has gained the most attention and popularity and it is able to deal with the issue of data sparsity (Koren et al., 2009). In addition, MF-based methods usually have greater capability than memory-based methods in capturing the similarity between users and the similarity between Web services, which therefore results in better QoS prediction performance.

However, most existing MF-based QoS prediction methods directly utilize -norm to measure the difference between the observed QoS values and the predicted ones (Zheng and Lyu, 2013; Lo et al., 2015; Xu et al., 2016; Xie et al., 2016; Su et al., 2016; Wu et al., 2018a, 2019a). It is well-known that -norm is sensitive to outliers (Zheng et al., 2012; Zhao et al., 2015; Meng and De La Torre, 2013; Cao et al., 2015; Xu et al., 2020; Zhu et al., 2017b). That is, the objective function value may be dominated by outliers during the -norm minimization process, which will lead to severe approximation deviation between the observed normal values and the predicted ones. As a result, without taking outliers into consideration, existing MF-based methods may not achieve satisfactory performance. In recent years, there are some explorations on enhancing the robustness of MF-based QoS prediction methods by replacing -norm with -norm (Zhu et al., 2018). Although -norm is more robust to outliers (Ke and Kanade, 2005; Eriksson and Van Den Hengel, 2010; Zheng et al., 2012; Meng et al., 2013), -norm-based objective function is much harder to optimize and the solution is also unstable (Xu et al., 2012; Meng and De La Torre, 2013). Moreover, -norm is still sensitive to outliers especially when outliers take significantly different values from the normal ones (Ding and Jiang, 2017; Xu et al., 2015). There are also some methods seeking to identify outliers explicitly by means of clustering algorithms (Zhu et al., 2010; Wu et al., 2019b; Hasny et al., 2016), which usually treat all the elements in the smallest cluster as outliers (Wu et al., 2015; He et al., 2014). However, it is difficult to choose a proper number of clusters, thus either some outliers may not be eliminated successfully or some normal values may be selected as outliers falsely.

To deal with the aforementioned issues, in this paper we propose a novel robust QoS prediction method under the matrix factorization framework. Our method chooses to measure the discrepancy between the observed QoS values and the predicted ones by Cauchy loss (Barron, 2019; Li et al., 2018) instead of the -norm loss or -norm loss. It has been shown that Cauchy loss is much more robust to outliers than the -norm loss and -norm loss (Li et al., 2018; Xu et al., 2015). Theoretically, Cauchy loss allows nearly half of the observations to be out of the normal range before it gives incorrect results (Mizera and Müller, 2002)

. Apparently, for a given QoS dataset, it is almost impossible that nearly half of the observations are outliers. Thus, Cauchy loss is sufficient for outlier modeling and has the potential to provide better prediction results. Note also that our method does not identify outliers explicitly, which reduces the risk of misclassification and makes our method more general and more robust. In other words, our method is resilient to outliers. Considering that the QoS value of a Web service observed by a specific user may change over time, it is essential to provide time-aware personalized QoS prediction results. To achieve this goal, we further extend our method under the tensor factorization framework by taking the temporal information into consideration.

In summary, the main contributions of this paper include:

  • First, we propose a robust Web service QoS prediction method with outlier resilience. Our method measures the discrepancy between the observed QoS values and the predicted ones by Cauchy loss, which is robust to outliers.

  • Second, we extend our method to provide time-aware QoS prediction results under the tensor factorization framework by taking the temporal information into consideration.

  • Third, we conduct extensive experiments on both static and dynamic datasets to evaluate the performance of our method. The results demonstrate that our method can achieve better performance than state-of-the-art baseline methods.

The remainder of this paper is organized as follows. Section 2 shows the unavoidability of outliers in QoS observations. Section 3 provides an overview on how matrix factorization can be employed for QoS prediction. Section 4 presents the detailed description of our method and its extension for time-aware QoS prediction. Section 5 reports the experimental results. Section 6 gives a brief review of the related work. This paper is finally concluded in Section 7.

(a) Response Time
(b) Throughput
(c) Response Time
(d) Throughput
Figure 1. The distribution of response time and throughput.

2. Unavoidability of QoS Outliers

Most existing QoS prediction methods assume that the QoS observations are reliable and rational. However, this assumption may not hold in the real world. This is because the observed QoS data can be affected by many factors. For example, there may be some malicious users submitting wrong QoS values deliberately. The service providers may also pretend to be service users and thus exaggerate the performance of their own Web services and depreciate the performance of their competitors’ Web services. In addition, the QoS values observed by users are largely dependent on the invocation environment such as network latency and server overload, which may lead some of the QoS values to deviate far from the normal range. In consideration of these complicated factors, we argue that it is highly possible that some of the QoS observations are outliers.

However, we are in lack of an oracle showing in advance which QoS observations are outliers. Here, we treat the rare extreme values which significantly differ from the remaining ones as outliers by following the definition in (Kim et al., 2016). To be more intuitive, in Figure 1 (a) and Figure 1 (b), we show the distribution of both response time and throughput of 100 Web services invoked by 3 randomly selected users from a publicly available dataset–WS-DREAM_dataset1222https://github.com/wsdream/wsdream-dataset/tree/master/dataset1. As can be seen, although a user tends to have different usage experiences on different Web services, most QoS values of these Web services observed by the three users fall into a normal range. For example, the response time mainly falls in the interval of . However, there are also some observations deviating far from the normal range. As shown in Figure 1 (a), the response time experienced by user 2 even reaches up to 14 seconds, which is far beyond the normal range. Needless to say, such kind of observations should be treated as outliers. We further demonstrate the distribution of response time and throughput of 3 Web services experienced by 100 different users in Figure 1 (c) and Figure 1 (d). It can be observed that although the usage experiences of a Web service can vary widely among different users, the QoS values of the same Web service observed by the majority of users tend to fall into a normal range. Whereas, there are also some observations taking values far beyond the normal range. These phenomena verify the rationality of treating extreme values as outliers and also reveal the unavoidability of outliers in QoS observations.

3. Preliminaries

Suppose that we are provided with a set of users and a set of Web services, then the QoS values between all users and Web services can be represented by a matrix whose entry denotes the QoS value of Web service observed by user . Obviously, it is both time-consuming and resource-consuming for each user to invoke all Web services to get the personalized QoS values. As a consequence, we typically have only partial observations between users and Web services, which means that lots of entries in are null. The goal of QoS prediction is to predict these null entries by exploiting the information contained in existing observations.

3.1. Problem Definition

Let denote the set of existing QoS observations, that is,

(1)

Then the problem of QoS prediction is defined as follows.

Problem Statement: Given a set of QoS observations , QoS prediction aims at predicting unknown QoS values by utilizing the information contained in .

3.2. Matrix Factorization for QoS Prediction

Matrix factorization is arguably the most popular model-based QoS prediction method at present (Zhang et al., 2019). Generally speaking, matrix factorization tries to factorize a given matrix into the product of several low-rank factor matrices. In the context of QoS prediction, the basic framework of MF-based methods is to factorize matrix into two low-rank factor matrices and , i.e., . Here, each row of represents the latent feature of a user, and each row of represents the latent feature of a Web service. The dimensionality of latent features is controlled by parameter . It is apparent that should be as close to as possible, which means that in MF-based methods, the QoS values are determined by users’ latent features and Web services’ latent features simultaneously. More specifically, the rows of and imply the contribution to the corresponding QoS values from user side and Web service side respectively (He et al., 2014). In order to make approximate well, there is no doubt that similar users and similar Web services should have similar latent features. Compared with memory-based QoS prediction methods that need to calculate similar users and similar Web services directly (e.g., calculating similarity according to Pearson correlation coefficient (PCC) (Zheng and Lyu, 2010; Zou et al., 2018)), MF-based methods are able to capture the similarity automatically, which lends more flexibility and generability to themselves.

Based on the discussions above, we can derive the following general objective function for MF-based QoS prediction methods:

(2)

where measures the degree of approximation between and , denotes the regularization term to avoid over-fitting, and represents the regularization coefficient.

The most widely adopted loss function in matrix factorization is the least square loss (i.e.,

-norm loss), which is also the most commonly used loss function in MF-based QoS prediction methods (Zheng and Lyu, 2013; Lo et al., 2015; Xu et al., 2016; Zhang et al., 2019; Zhu et al., 2018). In this setting, the specific objective function can be clearly given as follows:

(3)

where denotes the -norm which is calculated as the square root of the sum of squares of all entries, denotes the Hadamard product (i.e., entry-wise product), and denotes the indicator matrix whose entry indicates whether the QoS value of Web service has been observed by user or not. If user has the record of Web service , is set to 1; otherwise, it is set to 0.

The objective function based on -norm as in Eq. (3) is smooth and can be optimized by the gradient descent method (Gemulla et al., 2011). However, the -norm is sensitive to outliers (i.e., rare extreme values) (Zhao et al., 2015). When the given observations contain outliers, the residuals between these outliers’ corresponding entries in and their approximation entries in become huge due to the square operation. Therefore, when minimizing the objective function in Eq. (3), more priorities are given to these outliers, which unfortunately causes severe approximation deviation of the normal QoS values. As a result, the QoS prediction performance may degrade dramatically.

To make the model more robust to outliers, a common stategy is to replace -norm with -norm (Zhu et al., 2018; Wu et al., 2018b; Ke and Kanade, 2005; Eriksson and Van Den Hengel, 2010). Based on -norm, the objective function is formularized as below:

(4)

where denotes the -norm which is calculated as the sum of the absolute values of all entries. Although -norm is to some extent more robust to outliers than -norm, the objective function based on -norm as in Eq. (4) is a non-smooth function and it is much harder to optimize. What’s more, although the large residuals due to outliers are not squared in -norm, they may still be quite large relative to the normal ones and thus one would expect that they would influence the objective function as well (Ding and Jiang, 2017).

4. Our Method

As stated in the previous section, both -norm and -norm are sensitive to outliers. In order to make the MF-based methods more robust to outliers, we propose a novel QoS prediction method which utilizes Cauchy loss (Barron, 2019) as the measurement of the discrepancy between the observed QoS values and the predicted ones. It has been shown that Cauchy loss is resistant to outliers (Xu et al., 2015; Guan et al., 2017; Mizera and Müller, 2002). Thus our method is expected to be robust to outliers.

4.1. M-estimator

Before presenting the details of our method, we first introduce the concept of M-estimator. In robust statistics, M-estimators are a broad class of estimators, which represent the minima of particular loss functions

(Huber, 2011). Let denote the residual of the -th datum, i.e., the difference between the -th observation and its approximation. Then M-estimators try to optimize the following objective function:

(5)

where function gives the contribution of each residual to the objective function. A reasonable function should satisfy the following four properties (Fox, 2002):

  • , ;

  • , ;

  • ;

  • is non-decreasing in , i.e., , .

The influence function of is defined as its first-order derivative:

(6)

The influence function measures the influence of each datum on the value of the parameter estimate. For a robust M-estimator, it would be inferred that the influence of any single datum is insufficient to yield any significant offset (Xu et al., 2015). Ideally, a robust M-estimator should have a bounded influence function.

Both -norm loss and -norm loss satisfy the four properties required by M-estimators. For the estimator with , the influence function is , which means that the influence of a datum on the parameter estimate grows linearly as the error increases. This confirms the non-robusteness of estimator to outliers. Although the estimator with can reduce the influence of large errors due to its bounded influence function, it will still be affected by outliers since its influence function has no cut off point (Xu et al., 2015; Li et al., 2018) ( even when ). Besides, estimator is not stable because is not strictly convex in . It follows that the influence function of a robust M-estimator should not only be bounded but also be insensitive to the increase of errors ( when ). Cauchy estimator has been shown to possess such precious characteristics. The function of Cauchy estimator (i.e., Cauchy loss) is shown as follows:

(7)

where is a constant. The influence function is then calculated as

(8)

which takes value in the range of . Moreover, tends to zero when goes to infinity. This indicates that the influence function of Cauchy estimator is insensitive to the increase of errors. Therefore, Cauchy estimator is robust to outliers. A comparison of different M-estimators is illustrated in Figure 2.

(a) The Loss Function
(b) The Influence Function
Figure 2. Comparison of different M-estimators.

4.2. Model Formulation

In view of the robustness of Cauchy estimator, we choose Cauchy loss to construct the objective function of our method. Based on Cauchy loss, the objective function is derived as

(9)

where and denote the -th row of and the -th row of respectively, and represent the regularization coefficients.

The objective function in Eq. (9) can be efficiently optimized by the gradient descent method (Gemulla et al., 2011). Specifically, we choose to optimize and row by row. Then, we have the following update rules:

(10)
(11)

where and denote the learning rates for and , and

(12)
(13)

The overall optimization procedure of our method is presented in Algorithm 1, whose time complexity is shown in Theorem 1.

Theorem 1 ().

Let denote the number of iterations for Algorithm 1 to achieve convergence and let denote the number of available entries in , then the time complexity of Algorithm 1 is .

Proof.

The main time cost of Algorithm 1 lies in the updates of and . In each iteration, updating takes time and updating takes time. Since both and are less than , the time complexity of updating and can both be simplified as . Thus, the overall time complexity is of order . ∎

0:  , , , , , , ;
0:  , ;
1:  Randomly initialize and ;
2:  repeat
3:     for  to  do
4:        Update according to Eq. (10);
5:     for  to  do
6:        Update according to Eq. (11);
7:  until Convergence
8:  return  , ;
Algorithm 1 Algorithm for Static QoS Prediction

4.3. Extension for Time-aware QoS Prediction

As pointed out in (Zhang et al., 2011), the QoS performance of Web services is highly related to the invocation time because the service status (e.g., number of users) and the network environment (e.g., network speed) may change over time. Thus, it is essential to provide time-aware personalized QoS information to help users make service selection at runtime (Zhu et al., 2017a). In this part, we aim to extend our method in order to make it suitable for time-aware personalized QoS prediction.

To achieve the goal, we choose to extend our method under the tensor factorization framework. A tensor is a multidimensional or -way array (Kolda and Bader, 2009). An -way tensor is denoted as , which has indices () and its entries are denoted by . In this sense, a tensor can be treated as a generalized matrix and a matrix can also be treated as a two-way tensor.

For time-aware QoS prediction, we need to take the temporal information of QoS values into consideration. According to the definition of tensors, it is clear that we can model QoS observations with temporal information as a three-way tensor whose each entry represents the QoS value of Web service observed by user at time . Here, denotes the total number of time intervals. Accordingly, we can use an indicator tensor to show whether the QoS values have been observed or not. If user has the record of Web service at time , is set to 1; otherwise, its value is set to 0. To predict the unknown QoS values in , similar to MF-based methods, we first factorize to learn the latent features of users, Web services and contexts respectively, and then leverage the interaction among them to predict QoS values. Specifically, we adopt the canonical polyadic (CP) decomposition method (Rabanser et al., 2017) to factorize into three low-rank factor matrices , and . Then, is approximated in the following way:

(14)

where , and denote the -th column of , and respectively, and

represents the vector outer product. In this way, each entry

is approximated by

(15)

In the literature, there are some approaches (Zhang et al., 2011, 2014; Luo et al., 2019) resorting to tensor factorization for time-aware QoS prediction as well. However, none of these approaches have taken outliers into consideration. Besides, these approaches usually enforce nonnegative constraints on the factor matrices to promote the model’s interpretability. Following the same strategy, we also add nonnegative constraints to all the factor matrices , and . Then together with the Cauchy loss, we derive the objective function for time-aware QoS prediction as below:

(16)

where denotes the regularization coefficient for matrix .

Due to the nonnegative constraints, we cannot adopt the gradient descent method to optimize the objective function in Eq. (16) any more. Alternatively, we use the multiplicative updating (MU) algorithm (Lee and Seung, 2001) to solve Eq. (16). To be more specific, MU alternately updates , and with the other two being fixed in each iteration. Although the objective function in Eq. (16) is nonconvex over , and simultaneously, it is a convex function in each variable when the other two are fixed. Thus we can derive a closed-form update rule for each variable under the Karush-Kuhn-Tucker (KKT) conditions (Boyd and Vandenberghe, 2004). The detailed update rules are listed as follows:

(17)
(18)
(19)

In the above Equations, is defined as .

0:  , , , , , ;
0:  , , ;
1:  Randomly initialize , and ;
2:  repeat
3:     for  to  do
4:        Update according to Eq. (17);
5:     for  to  do
6:        Update according to Eq. (18);
7:     for  to  do
8:        Update according to Eq. (19);
9:  until Convergence
10:  return  , , ;
Algorithm 2 Algorithm for Time-aware QoS Prediction

Algorithm 2 summarizes the overall optimization procedure of our time-aware QoS prediction method. Since Algorithm 2 updates , and alternately and each update decreases the objective function value monotonically, it is guaranteed to converge to a local minimal solution. The time complexity of Algorithm 2 is shown in Theorem 2.

Theorem 2 ().

Let denote the number of iterations for Algorithm 2 to achieve convergence and let denote the number of available entries in , then the time complexity of Algorithm 2 is .

Proof.

The proof is similar to that of Theorem 1. In each iteration, it takes time to update , and . Therefore, the total time complexity of Algorithm 2 is of order . ∎

It is worth mentioning that in both Algorithm 1 and Algorithm 2, we do not detect outliers explicitly, thus our method will not suffer from the problem of misclassification, which indicates that our method is more resilient and more robust to outliers.

5. Experiments

In this section, we conduct a set of experiments on both static QoS prediction and time-aware QoS prediction to evaluate the efficiency and effectiveness of our method by comparing it with several state-of-the-art QoS prediction methods. We implement our method and all baseline methods in Python 3.7. And all the experiments are conducted on a server with two 2.4GHz Intel Xeon CPUs and 128GB main memory running Ubuntu 14.04.5 (64-bit).

5.1. Datasets

We conduct all experiments on a publicly available dataset collection–WS-DREAM333https://github.com/wsdream/wsdream-dataset, which was collected from real-world Web services. WS-DREAM contains both static and dynamic QoS datasets. The static dataset describes real-world QoS measurements, including both response time and throughput values, obtained from 339 users on 5825 Web services. The dynamic dataset describes real-world QoS measurements from 142 users on 4500 Web services over 64 consecutive time slices (at 15-minute interval). The dynamic dataset also includes records of both response time and throughput values. The statistics of the datasets are presented in Table 1.

Type QoS Attributes #User #Service #Time Range Mean
Static Response Time (s) 339 5825 - 0-20 0.9086
Throughput (kbps) 339 5825 - 0-1000 47.5617
Dynamic Response Time (s) 142 4500 64 0-20 3.1773
Throughput (kbps) 142 4500 64 0-6727 11.3449
Table 1. Statistics of QoS Data

5.2. Evaluation Metrics

The most commonly used evaluation metrics for QoS prediction include mean absolute error (MAE)

(Zheng et al., 2011) and root mean square error (RMSE) (Zheng et al., 2011). Let denote the set of QoS values to be predicted (i.e., is the testing set) and let , then MAE is calculated as

(20)

and RMSE is calculated as

(21)

where denotes the predicted value for the observation . For both MAE and RMSE, smaller values indicate better performance.

However, according to the definition of MAE and RMSE, we can see that both MAE and RMSE are sensitive to outliers, which means that if contains outliers, then MAE and RMSE cannot truely reflect the QoS prediction performance. For example, suppose that is an outlier, then in order to get small MAE value and RMSE value, the predicted should be close to rather than the normal QoS value. As thus, smaller MAE or RMSE value may not really indicate better performance. To overcome this limitation, we eliminate outliers from when calculating MAE and RMSE. Note that we do not have groundtruth labels for outliers. Therefore, we need to detect outliers from scratch. To achieve this goal, we employ the Forest (short for isolation forest) method (Liu et al., 2008, 2012)

for outlier detection.

Forest detects outliers purely based on the concept of isolation without employing any distance or density measure, which makes Forest quite efficient and robust. Forest will calculate an outlier score for each datum. The score takes value in the range of and larger value indicates more possibility to be outliers. Based on the outlier score, we can set the number of outliers flexibly. To intuitively show the effectiveness of Forest, we report the outlier detection results of a randomly selected Web service from the static dataset in Figure 3, where the outlier ratio is set to 0.05. As can be seen, Forest demonstrates good performance in outlier detection. It can detect both overinflated and underinflated outliers.

(a) Response Time
(b) Throughput
Figure 3. Exemplary outlier detection results by Forest.
QoS Attributes Methods MAE RMSE
Response Time MF 0.5334 0.4103 0.3534 0.3044 0.2921 0.2848 0.8407 0.6978 0.6195 0.5742 0.5565 0.5438
MF 0.4041 0.4037 0.4036 0.2815 0.2786 0.2798 0.6120 0.6106 0.6103 0.5590 0.5544 0.5505
CAP 0.3603 0.3521 0.3312 0.2282 0.2112 0.1881 0.6439 0.6640 0.6789 0.5815 0.5987 0.5821
TAP 0.3385 0.2843 0.2449 0.2477 0.2812 0.3189 0.5512 0.4985 0.4589 0.4687 0.5155 0.5665
DALF 0.3955 0.3439 0.3081 0.2496 0.2492 0.2397 0.7466 0.6779 0.5974 0.5471 0.5403 0.5388
CMF 0.1762 0.1524 0.1408 0.1153 0.1102 0.1085 0.3705 0.3599 0.3504 0.3106 0.2877 0.2699
Throughput MF 13.9730 12.3750 10.7753 7.8371 7.8255 7.8071 28.9608 26.8906 24.6608 19.6406 19.2831 18.6451
MF 16.5509 13.1105 10.7200 7.5736 7.3263 7.1115 33.8889 27.9648 23.6611 18.1316 17.9458 17.3698
CAP 16.4269 16.3125 16.1946 9.7147 8.6984 7.8516 32.9558 32.9334 32.9540 23.7955 22.2425 21.3711
TAP 22.1419 19.8273 17.8388 14.5786 14.8380 15.4028 43.4987 40.9533 38.8371 33.3052 32.4076 32.0935
DALF 13.1968 11.9619 10.6882 7.8156 7.7902 7.7771 27.8531 26.0299 24.4506 19.3523 18.9886 18.2965
CMF 8.4573 7.2501 6.4300 5.1865 5.1241 5.0078 24.9137 20.8927 18.8985 17.2916 17.1433 16.9388
Table 2. Performance Comparison with Different Training Ratios on Static Dataset (Best Results in Bold Numbers)

5.3. Baseline Methods

For ease of presentation, we name our method for static QoS prediction as CMF and our method for time-aware QoS prediction as CTF hereafter. For static QoS prediction, we compare CMF with the following five methods:

  • MF: MF denotes the basic MF-based QoS prediction method (Zheng and Lyu, 2013) and it measures the discrepancy between the observed QoS values and the predicted ones by -norm.

  • MF: MF is also an MF-based QoS prediction method (Zhu et al., 2018). However, it utilizes the -norm loss to construct the objective function. MF is expected to be more robust to outliers. Note that we implement MF a little differently from the original one proposed in (Zhu et al., 2018). In our implementation, we ignore the privacy and location information.

  • CAP: CAP is a credibility-aware QoS prediction method (Wu et al., 2015). It first employs two-phase -means clustering algorithm to identify untrustworthy users (i.e., outliers), and then predicts unknown QoS values based on the clustering information contributed by trustworthy users.

  • TAP: TAP is a trust-aware QoS prediction method (Su et al., 2017). It aims to provide reliable QoS prediction results via calculating the reputation of users by a beta reputation system and it identifies outliers based on -means clustering as well.

  • DALF: DALF is a data-aware latent factor model for QoS prediction (Wu et al., 2019b). It utilizes the density peaks based clustering algorithm (Rodriguez and Laio, 2014) to detect unreliable QoS data directly.

For time-aware QoS prediction, we compare our CTF with the following five methods:

  • NNCP: NNCP is a tensor-based time-aware QoS prediction method (Zhang et al., 2014). It is based on CP decomposition and it imposes nonnegative constraints on all the factor matrices.

  • BNLFT: BNLFT is a biased nonnegative tensor factorization model (Luo et al., 2019). It incorporates linear biases into the model for describing QoS fluctuations and it adds nonnegative constraints to the factor matrices as well.

  • WLRTF: WLRTF is an MLE (maximum likelihood estimation) based tensor factorization method (Chen et al., 2016). It models the noise of each datum as a mixture of Gaussian (MoG).

  • PLMF

    : PLMF is an LSTM (long short-term memory)

    (Hochreiter and Schmidhuber, 1997) based QoS prediction method (Xiong et al., 2018). PLMF can capture the dynamic latent representations of users and Web services.

  • TASR: TASR is a time-aware QoS prediction method (Ding et al., 2018). It integrates similarity-enhanced collaborative filtering model and the ARIMA model (a time series analysis model) (Box and Pierce, 1970).

QoS Attributes Methods MAE RMSE
Response Time MF 0.4080 0.3732 0.3533 0.3445 0.3306 0.3072 0.8040 0.7210 0.6711 0.6508 0.6021 0.5810
MF 0.3761 0.3390 0.3185 0.2972 0.2702 0.2525 0.7935 0.6903 0.6398 0.5918 0.5575 0.3744
CAP 0.4163 0.3657 0.3311 0.2997 0.2739 0.2413 0.9789 0.8375 0.7616 0.6817 0.6191 0.5258
TAP 0.4562 0.3788 0.3268 0.2703 0.2183 0.1649 1.1536 0.9393 0.8148 0.6475 0.4294 0.2478
DALF 0.3622 0.3217 0.3071 0.2890 0.2781 0.2480 0.7695 0.6728 0.6346 0.5975 0.5701 0.5132
CMF 0.2134 0.1758 0.1545 0.1384 0.1253 0.1019 0.6582 0.5001 0.4452 0.3811 0.3195 0.2347
Throughput MF 11.8832 10.7024 9.6776 9.0889 8.6373 8.1358 32.9795 28.5992 25.3608 22.9710 21.1042 18.5597
MF 12.3647 10.7403 9.8674 9.2223 8.7708 8.1667 32.9672 27.7982 24.4438 22.1691 20.2018 17.4015
CAP 18.2991 16.8273 15.5975 13.8889 13.6477 12.6762 45.9353 39.6390 35.5944 31.4784 29.2029 25.2830
TAP 22.0584 18.8479 16.9577 15.9026 15.1283 14.2151 58.5192 47.7490 41.6689 38.5700 35.2813 31.2779
DALF 11.8763 10.5724 9.1783 8.9276 8.6037 8.0449 32.8586 28.5797 24.8752 22.7428 20.9789 18.3713
CMF 8.3266 7.2138 6.5143 6.0463 5.5718 5.0177 30.5885 26.0933 22.9529 20.7105 17.8538 14.7925
Table 3. Performance Comparison with Different Outlier Ratios on Static Dataset (Best Results in Bold Numbers)

Although MF, CAP, TAP and DALF are able to deal with outliers to some extent for static QoS prediction, to our best knowledge, our method CTF is the first to take outliers into consideration for time-aware QoS prediction. It is also worth emphasizing that our method and all baseline methods (except for CAP, TAP and DALF) will not detect outliers explicitly when learning the prediction model. The reason for detecting outliers during the testing phase is to make MAE and RMSE able to truely reflect the QoS prediction performance. For all methods, outliers will be removed when calculating MAE and RMSE. In addition, in the experiments, we run each method 10 times and report the average results for fair comparison.

5.4. Experiments for Static QoS Prediction

We first conduct experiments for static QoS prediction.

5.4.1. Parameter Settings

In the experiments, for all baseline methods, we tune the corresponding parameters following the guidance of the original papers. As for our method CMF, on the response time dataset, the parameters are set as , , , and . On the throughput dataset, the parameters are set as , , , and . For MF, MF and DALF, the feature dimensionality is also set to 30.

5.4.2. Experimental Results

We first report the results by varying the training ratios in the range of . This is to simulate various prediction scenarios with different data sparsity. For example, when the training ratio is set to 0.1, then of the dataset will be used as training data and the rest will be used as testing data. As aforementioned, during the testing phase, outliers should be eliminated explicitly. Here we set the outlier ratio to 0.1, which means of the testing data that have large outlier scores will be removed when calculating MAE and RMSE. The detailed comparison results are presented in Table 2. As can be seen, our method CMF consistently shows better performance than all baseline methods. Moreover, the MAE and RMSE values obtained by our method are much smaller than those of baseline methods, especially on the response time dataset. For instance, CMF achieves more than performance promotion on response time over both MAE and RMSE. From Table 2, we can also see that MF, MF, DALF and CMF tend to obtain smaller MAE and RMSE values as the training ratio increases. This is desired because larger training ratio indicates that more QoS observations (i.e., more information) will be used to train the prediction model. However, CAP and TAP do not show this pattern, especially on the response time dataset. We can also observe that although CAP, TAP and DALF explicitly take outliers into consideration during the training phase, their performance is not satisfactory. The reason may be the misclassification of outliers. Since our method does not detect outliers directly during the training phase, it will not suffer from the misclassification issue. The resilience of our method to outliers makes it more robust.

(a) MAE
(b) RMSE
(c) MAE
(d) RMSE
Figure 4. Impact of dimensionality on CMF (with outlier ratio set to 0.02 and 0.1).
(a) MAE
(b) RMSE
(c) MAE
(d) RMSE
Figure 5. Impact of parameter on CMF (with outlier ratio set to 0.02 and 0.1).

We then report the results by varying the outlier ratios in the range of . In this experiment, the training ratio is fixed at 0.5. The results are shown in Table 3. From Table 3, we can see that our method still shows the best performance under different outlier ratios. It can also be observed that the MAE and RMSE values of all methods become smaller as the outlier ratio increases. This is reasonable because the larger the outlier ratio is, the more testing data with large outlier scores will be removed. Thus, the effects of outliers on the calculation of MAE and RMSE will be reduced accordingly. From Table 3, we can further obtain that with the increasing of outlier ratios, the performance promotion of CMF relative to MF increases from to over MAE and from to over RMSE on the response time dataset. On the throughput dataset, the performance promotion also increases from to over MAE and from to over RMSE. The increase of performance promotion verifies the necessity of removing outliers during the testing phase. It also verifies again the robustness of our proposed method.

5.4.3. Impact of Dimensionality

The parameter dimensionality controls the dimension of latent features in the factor matrices. To study the impact of , we vary its value from 10 to 80 with a step size of 10. In this experiment, the training ratio is fixed at 0.5 and the outlier ratio (denoted as ) is set to 0.02 and 0.1. The results are illustrated in Figure 4. As we can see, both MAE and RMSE take smaller values when dimensionality grows. This is because when takes larger values, more features of users and Web services will be captured, thus resulting in more accurate prediction results. We also observe on the throughput dataset that the performance tends to be stable when , which indicates that is sufficient for the factor matrices to well approximate the original matrix.

(a) MAE
(b) RMSE
(c) MAE
(d) RMSE
Figure 6. Impact of data sparsity on CMF (with outlier ratio set to 0.02 and 0.1).

5.4.4. Impact of Parameter

Recall that denotes the constant in the Cauchy loss. Here we study its impact on the performance of our method by varying its value in the range of . In this experiment, the training ratio is fixed at 0.5 and the outlier ratio is set to 0.02 and 0.1 as well. The results are illustrated in Figure 5. As can be seen, our method is sensitive to . This is due to that implicitly determines which data will be treated as outliers during the training phase. Thus we need to choose a proper to achieve the best performance. As shown in Figure 5, should take value around 1 on the response time dataset and around 20 on the throughput dataset to obtain accurate prediction results.

QoS Attributes Methods MAE RMSE
Response Time NNCP 1.0796 1.0536 1.0550 1.0424 1.0406 1.0392 2.6401 2.5797 2.5809 2.5574 2.5668 2.5616
BNLFT 1.0828 1.0575 1.0467 1.0368 1.0556 1.0403 2.6181 2.5809 2.5682 2.5559 2.5731 2.5582
WLRTF 1.0560 1.0437 1.0288 1.0299 1.0218 1.0274 2.6009 2.5706 2.5642 2.5566 2.5571 2.5491
PLMF 2.6133 2.5932 2.4054 2.2097 2.1266 2.0247 4.5582 4.3536 4.3294 4.1843 3.9818 3.8542
TASR 2.8188 2.7120 2.5591 2.1184 2.0066 1.8854 6.3872 6.1807 5.9552 5.0212 4.8000 4.5447
CTF 0.9215 0.8981 0.8890 0.8860 0.8766 0.8750 2.5865 2.5579 2.5548 2.5529 2.5517 2.5401
Throughput NNCP 1.5079 1.4342 1.4287 1.3761 1.3708 1.3761 4.9207 4.7019 4.6404 4.5080 4.4484 4.4968
BNLFT 1.4241 1.3935 1.3791 1.3856 1.3695 1.3613 4.6031 4.4685 4.4537 4.4128 4.3595 4.3493
WLRTF 2.9576 2.9568 2.9564 2.9561 2.9562 2.9537 4.9161 4.9160 4.9165 4.9153 4.9159 4.9095
PLMF 2.4712 2.2602 2.4459 2.3328 2.4655 2.2329 3.6705 3.8363 3.8455 3.8209 3.7541 3.5119
TASR 4.3265 3.6419 3.4736 2.8803 2.8258 2.7417 5.9152 5.1844 5.0034 4.3744 4.3142 4.2709
CTF 1.3567 1.1945 1.1225 0.9907 0.9889 0.9782 3.0436 2.9225 2.8576 2.7732 2.6978 2.6178
Table 4. Performance Comparison with Different Training Ratios on Dynamic Dataset (Best Results in Bold Numbers)

5.4.5. Impact of Data Sparsity

To evaluate the performance of our method comprehensively, it is also necessary to investigate the impact of the sparsity of training data. To this end, we vary the training ratio from 0.1 to 0.9 with a step size of 0.1. Apparently, different training ratio implies different data sparsity. In this experiment, we also set the outlier ratio to 0.02 and 0.1. The results are reported in Figure 6. From Figure 6, we see that as the training ratio increases (i.e., the sparsity of data decreases), more accurate results are obtained.

5.5. Experiments for Time-aware QoS Prediction

We now conduct experiments for time-aware QoS prediction.

5.5.1. Parameter Settings

In the experiments, we tune the parameters of all baseline methods following the guidance of the original papers. As for our method CTF, on the response time dataset, the parameters are set as and . is set to 10 when calculating MAE and 35 when calculating RMSE. On the throughput dataset, the parameters are set as and . is fixed at 5 for both MAE and RMSE. As for NNCP, BNLFT, WLRTF and PLMF, the feature dimensionality is also set to 15.

5.5.2. Experimental Results

We first report the results by varying the training ratios in the range of and fixing the outlier ratio at 0.1. The results are presented in Table 4. From Table 4, we can see that our method consistently shows better performance than all baseline methods on both datasets. The results verify the robustness of our method in the time-aware extension.

We then report the results by varying the outlier ratios in the range of and fixing the training ratio at 0.5. The results are shown in Table  5, from which we observe that our method achieves better performance under different outlier ratios, which is similar to the results on the static dataset.

QoS Attributes Methods MAE RMSE
Response Time NNCP 1.1846 1.1451 1.1069 1.0692 1.0521 1.0204 2.6740 2.6393 2.6023 2.5826 2.5805 2.5799
BNLFT 1.1647 1.1253 1.0871 1.0654 1.0475 0.9936 2.6499 2.6149 2.5771 2.5687 2.5659 2.5646
WLRTF 1.1436 1.1036 1.0562 1.0435 1.0261 0.9758 2.6438 2.6079 2.5696 2.5609 2.5584 2.5581
PLMF 2.6379 2.6011 2.5798 2.4241 2.3315 2.3162 5.2828 5.0571 4.7704 4.5917 4.2765 4.0729
TASR 2.5125 2.4326 2.3589 2.3363 2.3292 2.3019 5.4851 5.4510 5.4229 5.4018 5.3942 5.3877
CTF 1.0292 0.9813 0.9357 0.9105 0.8879 0.8448 2.6369 2.6015 2.5627 2.5564 2.5541 2.5503
Throughput NNCP 2.4853 2.0339 1.7508 1.5419 1.3926 1.0231 9.8925 7.6471 6.2757 5.2096 4.5026 2.8916
BNLFT 2.4335 1.9909 1.7163 1.5137 1.3693 1.0117 9.7267 7.4979 6.1664 5.1236 4.4319 2.8376
WLRTF 6.4309 4.8846 3.9224 3.3382 2.9562 2.0911 17.9461 11.6611 7.9882 5.9178 4.9156 3.1509
PLMF 5.3105 4.1556 3.0467 2.9632 2.3924 2.1807 13.7985 8.7995 6.6349 4.9651 3.8347 3.7906
TASR 5.7661 4.5595 3.8264 3.3965 3.1322 2.6317 14.8241 9.7143 6.8450 5.2958 4.6089 3.5886
CTF 2.2624 1.6385 1.3421 1.1437 1.0193 0.7323 9.5370 6.0759 4.3650 3.2415 2.7989 1.8020
Table 5. Performance Comparison with Different Outlier Ratios on Dynamic Dataset (Best Results in Bold Numbers)
(a) Static
(b) Dynamic
Figure 7. Runtime comparison on the response time dataset.

5.6. Efficiency Analysis

In this section, we investigate the runtime efficiency of our method. In this experiment, we fix the training ratio at 0.5 and the outlier ratio at 0.1. The runtime of different methods on the response time dataset is reported in Figure 7. On the static dataset, we can observe that CMF is very efficient. Its runtime is comparable to that of MF and MF. It also runs much faster than CAP, TAP and DALF. On the dynamic dataset, although CTF runs slower than PLMF and TASR, it is faster than BNLFT and WLRTF, and it is comparable to NNCP.

6. Related Work

6.1. Collaborative QoS Prediction

Most existing QoS prediction methods fall into collaborative filtering methods (Wu et al., 2015), which can be further divided into two categories: memory-based methods (Zheng et al., 2011; Cao et al., 2013; Jiang et al., 2011; Breese et al., 1998; Sarwar et al., 2001) and model-based methods (Wu et al., 2018b; Rangarajan, 2018; Yang et al., 2018; Xie et al., 2016; Su et al., 2016). Memory-based methods predict unknown QoS values by employing the neighbourhood information of similar users and similar Web services (Zheng and Lyu, 2010), which further leads to user-based methods (Breese et al., 1998), service-based methods (Sarwar et al., 2001) and hybrid methods (Zheng et al., 2011; Cao et al., 2013; Jiang et al., 2011) that systematically combine the user-based methods and service-based methods. It is well known that memory-based methods usually suffer from the data sparsity problem (Zhang et al., 2019; Zheng and Lyu, 2013) due to limited number of Web services a single user will invoke. Model-based methods are able to deal with the problem of data sparsity, thus they have gained the most popularity (Zhang et al., 2019). Model-based methods usually train a predefined prediction model based on existing QoS observations and then predict missing QoS values. For example, Wu et al. (Wu et al., 2017) propose to train a factorization machine model for QoS prediction. Luo et al. (Luo et al., 2015)

introduce fuzzy neural networks and adaptive dynamic programming to predict QoS values. Matrix factorization is also a model-based technique and it has obtained the most attention

(Zheng and Lyu, 2013; Lo et al., 2015; Xie et al., 2016; Su et al., 2016; Zhang et al., 2019). MF-based methods factorize the user-service matrix into two low-rank factor matrices with one factor matrix capturing the latent representations of users and another revealing the latent representations of Web services. Therefore, MF-based methods are able to automatically model the contributions to a specific QoS value from user side and service side simultaneously, which usually results in better prediction performance. In addition, MF-based methods possess high flexibility of incorporating side information such as location (He et al., 2014), contexts (Wu et al., 2018a, b) and privacy (Liu et al., 2018). MF-based methods can also be easily generalized for time-aware QoS prediction under the tensor factorization framework (Zhang et al., 2011, 2014; Luo et al., 2019; Wang et al., 2019). There are also a few other kinds of time-aware QoS prediction methods like time series model based methods (Ding et al., 2018; Amin et al., 2012) and neural networks based methods (Xiong et al., 2018).

6.2. Reliable QoS Prediction

Although there are various QoS prediction methods, few of them have taken outliers into consideration. However, as analyzed in Section 2, some QoS observations indeed should be treated as outliers. Thus, the performance of existing methods may not be reliable. For example, most existing MF-based QoS prediction methods directly utilize -norm to measure the discrepancy between the observed QoS values and the predicted ones (Zheng and Lyu, 2013; Lo et al., 2015; Xu et al., 2016; Xie et al., 2016; Su et al., 2016; Wu et al., 2018a). It is widely accepted that -norm is not robust to outliers (Zheng et al., 2012; Zhao et al., 2015; Meng and De La Torre, 2013; Cao et al., 2015; Xu et al., 2020; Zhu et al., 2017b). As a consequence, the performance of MF-based methods may be severely influenced when QoS observations contain outliers.

In order to obtain reliable QoS prediction results, it is necessary to take outliers into consideration. One popular method to reduce the effects of outliers is replacing -norm with -norm because -norm is more robust to outliers (Ke and Kanade, 2005; Eriksson and Van Den Hengel, 2010; Zheng et al., 2012; Meng et al., 2013). For example, an -norm low-rank MF-based QoS prediction method is introduced in (Zhu et al., 2018). However, -norm-based objective function is non-smooth and thus much harder to optimize (Xu et al., 2012; Meng and De La Torre, 2013). Besides, -norm is still sensitive to outliers especially when outliers are far beyond the normal range of QoS values (Ding and Jiang, 2017; Xu et al., 2015).

Another line of reliable QoS prediction is detecting outliers explicitly based on clustering algorithms. In (Wu et al., 2015), Wu et al. propose a credibility-aware QoS prediction method, which employs two-phase -means clustering algorithm to identify untrustworthy users (i.e., outliers). Su et al. (Su et al., 2017) propose a trust-aware QoS prediction method, which provides reliable QoS prediction results via calculating the reputation of users by a beta reputation system and identifies outliers based on -means clustering as well. In (Wu et al., 2019b), a data-aware latent factor model is introduced, which utilizes the density peaks based clustering algorithm (Rodriguez and Laio, 2014) to detect unreliable QoS values. However, it is difficult to choose a proper number of clusters, thus either some outliers may not be eliminated successfully or some normal values may be selected as outliers falsely. Our method does not detect outliers explicitly, therefore it will not suffer from the misclassification issue.

7. Conclusion

In this paper, we have proposed a novel robust QoS prediction method, which utilizes Cauchy loss to measure the discrepancy between the observed QoS values and the predicted ones. Owing to the robustness of Cauchy loss, our method is resilient to outliers. That is, there is no need to detect outliers explicitly. Therefore, our method will not suffer from the problem of misclassification. Considering that the QoS performance may change over time, we have further extended our method to make it suitable for time-aware QoS prediction. To evaluate the efficiency and effectiveness of our method, we have conducted extensive experiments on both static and dynamic datasets. Experimental results have demonstrated that our method can achieve better performance than existing methods.

References

  • A. Amin, L. Grunske, and A. Colman (2012) An automated approach to forecasting qos attributes based on linear and non-linear time series modeling. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 130–139. Cited by: §6.1.
  • J. T. Barron (2019) A general and adaptive robust loss function. In

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    ,
    pp. 4331–4339. Cited by: §1, §4.
  • A. Bouguettaya, M. Singh, M. Huhns, Q. Z. Sheng, H. Dong, Q. Yu, A. G. Neiat, S. Mistry, B. Benatallah, B. Medjahed, et al. (2017) A service computing manifesto: the next 10 years. Communications of the ACM 60 (4), pp. 64–72. Cited by: §1.
  • G. E. Box and D. A. Pierce (1970) Distribution of residual autocorrelations in autoregressive-integrated moving average time series models. Journal of the American Statistical Association 65 (332), pp. 1509–1526. Cited by: 5th item.
  • S. Boyd and L. Vandenberghe (2004) Convex optimization. Cambridge University Press. Cited by: §4.3.
  • J. S. Breese, D. Heckerman, and C. Kadie (1998) Empirical analysis of predictive algorithms for collaborative filtering. In

    Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence (UAI)

    ,
    pp. 43–52. Cited by: §6.1.
  • J. Cao, Z. Wu, Y. Wang, and Y. Zhuang (2013) Hybrid collaborative filtering algorithm for bidirectional web service recommendation. Knowledge and Information Systems 36 (3), pp. 607–627. Cited by: §6.1.
  • X. Cao, Y. Chen, Q. Zhao, D. Meng, Y. Wang, D. Wang, and Z. Xu (2015) Low-rank matrix factorization under general mixture noise distributions. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1493–1501. Cited by: §1, §6.2.
  • X. Chen, Z. Han, Y. Wang, Q. Zhao, D. Meng, and Y. Tang (2016) Robust tensor factorization with unknown noise. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5213–5221. Cited by: 3rd item.
  • Z. Chen, L. Shen, F. Li, D. You, and J. P. B. Mapetu (2020) Web service qos prediction: when collaborative filtering meets data fluctuating in big-range. World Wide Web, pp. 1–26. Cited by: §1.
  • C. Ding and B. Jiang (2017) L1-norm error function robustness and outlier regularization. arXiv preprint arXiv:1705.09954. Cited by: §1, §3.2, §6.2.
  • S. Ding, Y. Li, D. Wu, Y. Zhang, and S. Yang (2018) Time-aware cloud service recommendation using similarity-enhanced collaborative filtering and arima model. Decision Support Systems 107, pp. 103–115. Cited by: 5th item, §6.1.
  • J. El Haddad, M. Manouvrier, and M. Rukoz (2010) TQoS: transactional and qos-aware selection algorithm for automatic web service composition. IEEE Transactions on Services Computing (1), pp. 73–85. Cited by: §1.
  • A. Eriksson and A. Van Den Hengel (2010) Efficient computation of robust low-rank matrix approximations in the presence of missing data using the l 1 norm. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 771–778. Cited by: §1, §3.2, §6.2.
  • J. Fox (2002) An r and s-plus companion to applied regression. Sage. Cited by: §4.1.
  • R. Gemulla, E. Nijkamp, P. J. Haas, and Y. Sismanis (2011)

    Large-scale matrix factorization with distributed stochastic gradient descent

    .
    In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (SIGKDD), pp. 69–77. Cited by: §3.2, §4.2.
  • N. Guan, T. Liu, Y. Zhang, D. Tao, and L. S. Davis (2017) Truncated cauchy non-negative matrix factorization. IEEE Transactions on Pattern Analysis and Machine Intelligence 41 (1), pp. 246–259. Cited by: §4.
  • F. Hasny, S. Mensah, D. Yi, C. Li, and R. Zhang (2016) Predicting the quality of web services based on user stability. In 2016 IEEE International Conference on Services Computing (SCC), pp. 860–863. Cited by: §1.
  • P. He, J. Zhu, Z. Zheng, J. Xu, and M. R. Lyu (2014) Location-based hierarchical matrix factorization for web service recommendation. In 2014 IEEE International Conference on Web Services (ICWS), pp. 297–304. Cited by: §1, §3.2, §6.1.
  • S. Hochreiter and J. Schmidhuber (1997) Long short-term memory. Neural Computation 9 (8), pp. 1735–1780. Cited by: 4th item.
  • P. J. Huber (2011) Robust statistics. Springer. Cited by: §4.1.
  • Y. Jiang, J. Liu, M. Tang, and X. Liu (2011) An effective web service recommendation method based on personalized collaborative filtering. In 2011 IEEE International Conference on Web Services (ICWS), pp. 211–218. Cited by: §6.1.
  • Q. Ke and T. Kanade (2005) Robust l/sub 1/norm factorization in the presence of outliers and missing data by alternative convex programming. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 1, pp. 739–746. Cited by: §1, §3.2, §6.2.
  • M. Kim, B. Oh, J. Jung, and K. Lee (2016) Outlier-robust web service selection based on a probabilistic qos model. International Journal of Web and Grid Services 12 (2), pp. 162–181. Cited by: §2.
  • T. G. Kolda and B. W. Bader (2009) Tensor decompositions and applications. SIAM Review 51 (3), pp. 455–500. Cited by: §4.3.
  • Y. Koren, R. Bell, and C. Volinsky (2009) Matrix factorization techniques for recommender systems. Computer (8), pp. 30–37. Cited by: §1.
  • D. D. Lee and H. S. Seung (2001) Algorithms for non-negative matrix factorization. In Advances in Neural Information Processing Systems (NIPS), pp. 556–562. Cited by: §4.3.
  • X. Li, Q. Lu, Y. Dong, and D. Tao (2018) Robust subspace clustering by cauchy loss function. IEEE Transactions on Neural Networks and Learning Systems 30 (7), pp. 2067–2078. Cited by: §1, §4.1.
  • A. Liu, X. Shen, Z. Li, G. Liu, J. Xu, L. Zhao, K. Zheng, and S. Shang (2018) Differential private collaborative web services qos prediction. World Wide Web, pp. 1–24. Cited by: §6.1.
  • F. T. Liu, K. M. Ting, and Z. Zhou (2008) Isolation forest. In 2008 Eighth IEEE International Conference on Data Mining (ICDM), pp. 413–422. Cited by: §5.2.
  • F. T. Liu, K. M. Ting, and Z. Zhou (2012)

    Isolation-based anomaly detection

    .
    ACM Transactions on Knowledge Discovery from Data (TKDD) 6 (1), pp. 3. Cited by: §5.2.
  • W. Lo, J. Yin, Y. Li, and Z. Wu (2015) Efficient web service qos prediction using local neighborhood matrix factorization. Engineering Applications of Artificial Intelligence 38, pp. 14–23. Cited by: §1, §3.2, §6.1, §6.2.
  • X. Luo, H. Wu, H. Yuan, and M. Zhou (2019) Temporal pattern-aware qos prediction via biased non-negative latent factorization of tensors. IEEE Transactions on Cybernetics, pp. 1–12. Cited by: §4.3, 2nd item, §6.1.
  • X. Luo, Y. Lv, R. Li, and Y. Chen (2015) Web service qos prediction based on adaptive dynamic programming using fuzzy neural networks for cloud services. IEEE Access 3, pp. 2260–2269. Cited by: §6.1.
  • D. Meng and F. De La Torre (2013) Robust matrix factorization with unknown noise. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1337–1344. Cited by: §1, §6.2, §6.2.
  • D. Meng, Z. Xu, L. Zhang, and J. Zhao (2013) A cyclic weighted median method for l1 low-rank matrix factorization with missing entries. In Twenty-Seventh AAAI Conference on Artificial Intelligence (AAAI), pp. 704–710. Cited by: §1, §6.2.
  • I. Mizera and C. H. Müller (2002) Breakdown points of cauchy regression-scale estimators. Statistics & Probability Letters 57 (1), pp. 79–89. Cited by: §1, §4.
  • M. P. Papazoglou, P. Traverso, S. Dustdar, and F. Leymann (2007) Service-oriented computing: state of the art and research challenges. Computer 40 (11), pp. 38–45. Cited by: §1.
  • S. Rabanser, O. Shchur, and S. Günnemann (2017)

    Introduction to tensor decompositions and their applications in machine learning

    .
    arXiv preprint arXiv:1711.10781. Cited by: §4.3.
  • S. Rangarajan (2018) Qos-based web service discovery and selection using machine learning. arXiv preprint arXiv:1807.01439. Cited by: §6.1.
  • A. Rodriguez and A. Laio (2014) Clustering by fast search and find of density peaks. Science 344 (6191), pp. 1492–1496. Cited by: 5th item, §6.2.
  • B. M. Sarwar, G. Karypis, J. A. Konstan, J. Riedl, et al. (2001) Item-based collaborative filtering recommendation algorithms. WWW 1, pp. 285–295. Cited by: §6.1.
  • L. Shao, J. Zhang, Y. Wei, J. Zhao, B. Xie, and H. Mei (2007) Personalized qos prediction for web services via collaborative filtering. In IEEE International Conference on Web Services (ICWS), pp. 439–446. Cited by: §1.
  • K. Su, L. Ma, B. Xiao, and H. Zhang (2016) Web service qos prediction by neighbor information combined non-negative matrix factorization. Journal of Intelligent & Fuzzy Systems 30 (6), pp. 3593–3604. Cited by: §1, §6.1, §6.2.
  • K. Su, B. Xiao, B. Liu, H. Zhang, and Z. Zhang (2017) TAP: a personalized trust-aware qos prediction approach for web service recommendation. Knowledge-Based Systems 115, pp. 55–65. Cited by: 4th item, §6.2.
  • Q. Wang, M. Chen, M. Shang, and X. Luo (2019) A momentum-incorporated latent factorization of tensors model for temporal-aware qos missing data prediction. Neurocomputing 344, pp. 299–307. Cited by: §6.1.
  • C. Wu, W. Qiu, Z. Zheng, X. Wang, and X. Yang (2015)

    QoS prediction of web services based on two-phase k-means clustering

    .
    In 2015 IEEE International Conference on Web Services (ICWS), pp. 161–168. Cited by: §1, §1, §1, 3rd item, §6.1, §6.2.
  • D. Wu, Q. He, X. Luo, M. Shang, Y. He, and G. Wang (2019a) A posterior-neighborhood-regularized latent factor model for highly accurate web service qos prediction. IEEE Transactions on Services Computing. Cited by: §1.
  • D. Wu, X. Luo, M. Shang, Y. He, G. Wang, and X. Wu (2019b) A data-aware latent factor model for web service qos prediction. In Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD), pp. 384–399. Cited by: §1, 5th item, §6.2.
  • H. Wu, K. Yue, B. Li, B. Zhang, and C. Hsu (2018a) Collaborative qos prediction with context-sensitive matrix factorization. Future Generation Computer Systems 82, pp. 669–678. Cited by: §1, §6.1, §6.2.
  • H. Wu, Z. Zhang, J. Luo, K. Yue, and C. Hsu (2018b) Multiple attributes qos prediction via deep neural model with contexts. IEEE Transactions on Services Computing. Cited by: §3.2, §6.1.
  • Y. Wu, F. Xie, L. Chen, C. Chen, and Z. Zheng (2017) An embedding based factorization machine approach for web service qos prediction. In International Conference on Service-Oriented Computing (ICSOC), pp. 272–286. Cited by: §6.1.
  • Q. Xie, S. Zhao, Z. Zheng, J. Zhu, and M. R. Lyu (2016) Asymmetric correlation regularized matrix factorization for web service recommendation. In 2016 IEEE International Conference on Web Services (ICWS), pp. 204–211. Cited by: §1, §6.1, §6.2.
  • R. Xiong, J. Wang, Z. Li, B. Li, and P. C. Hung (2018) Personalized lstm based matrix factorization for online qos prediction. In 2018 IEEE International Conference on Web Services (ICWS), pp. 34–41. Cited by: 4th item, §6.1.
  • C. Xu, D. Tao, and C. Xu (2015) Multi-view intact space learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 37 (12), pp. 2531–2544. Cited by: §1, §1, §4.1, §4.1, §4, §6.2.
  • H. Xu, C. Caramanis, and S. Mannor (2012) Sparse algorithms are not stable: a no-free-lunch theorem. IEEE Transactions on Pattern Analysis and Machine Intelligence 34 (1), pp. 187–193. Cited by: §1, §6.2.
  • J. Xu, Z. Zheng, and M. R. Lyu (2016) Web service personalized quality of service prediction via reputation-based matrix factorization. IEEE Transactions on Reliability 65 (1), pp. 28–37. Cited by: §1, §3.2, §6.2.
  • S. Xu, C. Zhang, and J. Zhang (2020)

    Adaptive quantile low-rank matrix factorization

    .
    Pattern Recognition, pp. 107310. Cited by: §1, §6.2.
  • Y. Yang, Z. Zheng, X. Niu, M. Tang, Y. Lu, and X. Liao (2018) A location-based factorization machine model for web service qos prediction. IEEE Transactions on Services Computing. Cited by: §6.1.
  • W. Zhang, H. Sun, X. Liu, and X. Guo (2014) Temporal qos-aware web service recommendation via non-negative tensor factorization. In Proceedings of the 23rd International Conference on World Wide Web (WWW), pp. 585–596. Cited by: §4.3, 1st item, §6.1.
  • Y. Zhang, Z. Zheng, and M. R. Lyu (2011) WSPred: a time-aware personalized qos prediction framework for web services. In 2011 IEEE 22nd International Symposium on Software Reliability Engineering (ISSRE), pp. 210–219. Cited by: §1, §4.3, §4.3, §6.1.
  • Y. Zhang, K. Wang, Q. He, F. Chen, S. Deng, Z. Zheng, and Y. Yang (2019) Covering-based web service quality prediction via neighborhood-aware matrix factorization. IEEE Transactions on Services Computing. Cited by: §1, §1, §3.2, §3.2, §6.1.
  • Q. Zhao, D. Meng, Z. Xu, W. Zuo, and Y. Yan (2015) {}-Norm low-rank matrix factorization by variational bayesian method. IEEE Transactions on Neural Networks and Learning Systems 26 (4), pp. 825–839. Cited by: §1, §3.2, §6.2.
  • Y. Zheng, G. Liu, S. Sugimoto, S. Yan, and M. Okutomi (2012) Practical low-rank matrix approximation under robust l 1-norm. In 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1410–1417. Cited by: §1, §6.2, §6.2.
  • Z. Zheng and M. R. Lyu (2010) Collaborative reliability prediction of service-oriented systems. In 2010 ACM/IEEE 32nd International Conference on Software Engineering (ICSE), Vol. 1, pp. 35–44. Cited by: §3.2, §6.1.
  • Z. Zheng and M. R. Lyu (2013) Personalized reliability prediction of web services. ACM Transactions on Software Engineering and Methodology (TOSEM) 22 (2), pp. 12. Cited by: §1, §1, §3.2, 1st item, §6.1, §6.2.
  • Z. Zheng, H. Ma, M. R. Lyu, and I. King (2011) Qos-aware web service recommendation by collaborative filtering. IEEE Transactions on Services Computing 4 (2), pp. 140–152. Cited by: §5.2, §6.1.
  • J. Zhu, P. He, Z. Zheng, and M. R. Lyu (2017a) Online qos prediction for runtime service adaptation via adaptive matrix factorization. IEEE Transactions on Parallel and Distributed Systems 28 (10), pp. 2911–2924. Cited by: §4.3.
  • R. Zhu, D. Niu, and Z. Li (2017b) Robust web service recommendation via quantile matrix factorization. In IEEE 2017 International Conference on Computer Communications (INFOCOM), pp. 1–9. Cited by: §1, §6.2.
  • X. Zhu, X. Jing, D. Wu, Z. He, J. Cao, D. Yue, and L. Wang (2018) Similarity-maintaining privacy preservation and location-aware low-rank matrix factorization for qos prediction based web service recommendation. IEEE Transactions on Services Computing. Cited by: §1, §3.2, §3.2, 2nd item, §6.2.
  • Z. Zhu, H. Yuan, J. Song, J. Bi, and G. Liu (2010) WS-scan: a effective approach for web services clustering. In 2010 International Conference on Computer Application and System Modeling (ICCASM), Vol. 5, pp. 618–622. Cited by: §1.
  • G. Zou, M. Jiang, S. Niu, H. Wu, S. Pang, and Y. Gan (2018) QoS-aware web service recommendation with reinforced collaborative filtering. In International Conference on Service-Oriented Computing (ICSOC), pp. 430–445. Cited by: §1, §3.2.