DeepAI

# Quantifying Interpretability of Arbitrary Machine Learning Models Through Functional Decomposition

To obtain interpretable machine learning models, either interpretable models are constructed from the outset - e.g. shallow decision trees, rule lists, or sparse generalized linear models - or post-hoc interpretation methods - e.g. partial dependence or ALE plots - are employed. Both approaches have disadvantages. While the former can restrict the hypothesis space too conservatively, leading to potentially suboptimal solutions, the latter can produce too verbose or misleading results if the resulting model is too complex, especially w.r.t. feature interactions. We propose to make the compromise between predictive power and interpretability explicit by quantifying the complexity / interpretability of machine learning models. Based on functional decomposition, we propose measures of number of features used, interaction strength and main effect complexity. We show that post-hoc interpretation of models that minimize the three measures becomes more reliable and compact. Furthermore, we demonstrate the application of such measures in a multi-objective optimization approach which considers predictive power and interpretability at the same time.

• 11 publications
• 20 publications
• 79 publications
06/17/2021

### Interpretable Machine Learning Classifiers for Brain Tumour Survival Prediction

Prediction of survival in patients diagnosed with a brain tumour is chal...
06/10/2016

### The Mythos of Model Interpretability

Supervised machine learning models boast remarkable predictive capabilit...
03/01/2021

### Interpretable Artificial Intelligence through the Lens of Feature Interaction

Interpretation of deep learning models is a very challenging problem bec...
06/18/2021

### It's FLAN time! Summing feature-wise latent representations for interpretability

Interpretability has become a necessary feature for machine learning mod...
11/16/2021

### SMACE: A New Method for the Interpretability of Composite Decision Systems

Interpretability is a pressing issue for decision systems. Many post hoc...
01/14/2019

### Interpretable machine learning: definitions, methods, and applications

Machine-learning models have demonstrated great success in learning comp...
07/08/2019

### Optimal Explanations of Linear Models

When predictive models are used to support complex and important decisio...

## 1 Introduction

Machine learning models are optimized for predictive performance, but it is often required to understand models in order to e.g. debug them, gain trust in the predictions and satisfy regulatory requirements. Therefore, performance often has to be traded off for interpretability. In areas such as life sciences and social sciences, it is common to restrict model selection to interpretable models such as (generalized) linear regression models and decision trees

[23]

. This often relies on an intuitive notion of interpretability, leading to an avoidance of ”black boxes” such as tree ensembles and neural networks

[5]. A restriction to structurally simpler models has the drawback that better performing models are often excluded a priori from model selection. An alternative is to allow any model and apply post-hoc interpretation methods to explain model behavior and predictions. Interpretation methods quantify effects that features have on predictions, compute feature importances or explain individual predictions, see [24] for an overview. While model-agnostic post-hoc interpretation methods can – in general – be used regardless of model complexity, their reliability and compactness deteriorates when models use a high number of features, have strong feature interactions and complex feature main effects.

Model-agnostic interpretability measures are needed to make the compromise between interpretability and predictive performance explicit when selecting models [31, 5]. Instead of fixing the trade-off by preselecting an interpretable model class, model-agnostic measures would allow informed model selection with the desired balance between interpretability and predictive performance [13]. Interpretability is not well defined [23] and depends on user preferences and domain [30, 14, 20, 31]. This supports the conclusion in [4] that we cannot summarize interpretability with a single metric.

#### 1.0.1 Contributions.

We propose three model-agnostic measures of machine learning model interpretability. The measures can be used to compare trained models or to explicitly optimize interpretability during hyperparameter tuning and model selection. First we review related work on interpretability measures and the background of functional decomposition, on which our proposed measures are based. For the

number of features used

by the model, we propose an estimation heuristic. Based on the decomposition of the prediction function, we suggest measures for

interaction strength and for average complexity of the feature main effects. We argue that minimizing these three measures improves the reliability and compactness of post-hoc interpretation methods. Finally, we illustrate the use of our proposed measures in multi-objective optimization and discuss implications of interpretability measures for the field of interpretable machine learning.

## 2 Related Work and Background

In this section we introduce the notation, review related work and describe the functional decomposition on which we base the proposed complexity measures.

#### 2.0.1 Notation:

We consider machine learning prediction functions , where

is a p-dimensional feature vector and

is the prediction (e.g. regression output or a classification score). For the decomposition of this function, we write , , to denote a function that maps a vector with a subset of features to a marginal prediction. If subset contains a single feature, we write . We refer to the training data for the machine learning model with the tuples and refer to the value of the j-th feature from the i-th instance as . We write

to refer to the j-th feature as a random variable.

### 2.1 Interpretability Measures

This section provides a non-exhaustive overview of approaches for measuring and optimizing interpretability. Many measures of interpretability are model-specific, i.e. only models of the same class can be compared (e.g. decision trees). Model size is often used as a measure for interpretability (e.g. number of decision rules, tree depth, …) [20, 31, 4, 35]. Akaikes Information Criterion (AIC) [1] and the Bayesian Information Criterion (BIC) [33]

are more widely applicable measures for the trade-off between goodness of fit and degrees of freedom. AIC and BIC fix a certain compromise between interpretability and performance, and consider only one dimension of interpretability, the degrees of freedom. In

[36] the authors propose an interpretability evaluation model that considers the (model-specific) structural complexity of machine learning models. In additive models (e.g. linear regression), the number of features is often used as a measure of interpretability [32]. In [26] the authors propose model-agnostic measures of model stability, based on the semantic similarity of predictions when the model is re-trained on different subsamples of the training data. Similar to our approach, the measures are based on the predictions, not on the structure of the model.

In [27] the authors propose explanation fidelity and explanation stability metrics of local explanation models [29]

. They propose to incorporate the metrics as a regularizer into the loss function of a neural network to simultaneously optimize for predictive performance and higher quality of local explanations. Their local explainability metrics complement ours, since we consider global model properties.

Further approaches measure interpretability as the usability of a (interpretable) model to support a human in a task, usually measured in a survey as response time, correctness of the response and task difficulty [36, 20, 10]. In [15] an interpretability measure based on runtime operation count is proposed and evaluated in user studies.

### 2.2 Functional Decomposition

Any high-dimensional prediction function can be decomposed into a sum of components with increasing dimensionality: an intercept, first-order feature effects, second-order effects and so on up to the p-th order effect:

 f(x)= Interceptf0+1st order effectsp∑j=1fj(xj)+2nd order effectsp∑j

This decomposition is only unique with additional constraints regarding the components. Stone [34] suggested orthogonality constraints and approximating the prediction function with weighted integrals. Hooker [18]

defined centering, orthogonality and variance decomposition as desirable properties, resulting in unique and hierarchically orthogonal components under the correlation inner product.

Accumulated Local Effects (ALE) were proposed in [3] as a tool for visualizing feature effects (e.g. Figure 1) and as an alternative unique decomposition of the prediction function with components . The ALE decomposition is unique under an orthogonality-like property further described in [3].

The ALE main effect of a feature for a prediction function is defined as

 fj,ALE(xj)=∫xjz0,jE[δf(X1,…,Xp)δXj∣∣∣Xj=zj]dzj−cj (2)

Here, is a lower bound of (usually the minimum observed value of ) and the expectation is computed conditional on the value for and over the marginal distribution of all other features. The constant is chosen so that the mean of with respect to the marginal distribution of is zero. The ALE main effects are defined as the gradients of with respect to the features, but are estimated with finite differences, i.e. access to the gradients of the model is not required. For the estimation we refer to [3]. We base our proposed measures on the ALE decomposition, because ALE are computationally cheap (worst case for a feature main effect), the effects can be computed sequentially instead of simultaneously as in [18] and, most importantly, ALEs do not require knowledge of the joint data distribution. Additionally, ALE have software implementations [25, 2].

## 3 Functional Complexity

In this section we motivate complexity measures based on functional decomposition. Based on Equation 1, we decompose the prediction function into a constant (estimated as ), plus the main effects (estimate with ALE), and a remainder term containing interactions (difference between full model and + main effects).

 f(x)= f0+p∑j=1MEC: How complex?fj,ALE(xj)+IAS: Interaction strength?IA(x)NF: How many features were used? (3)

This arrangement of components emphasizes a decomposition of the prediction function into a main effect model and an interaction remainder. The main effect model itself can be used as a prediction function and we can analyze how well it approximates , which is the idea behind the interaction measure IAS. The average main effect complexity (MEC) captures how many parameters are needed to describe the one-dimensional main effects on average. The number of features used (NF) describes how many features were used in the full prediction function.

### 3.1 Number of Features (NF)

We propose an approach based on feature permutation to determine how many features are used by a model. We regard features as ”used” by the model when changing a feature changes the prediction, which may differ from the numbers of features available during training.

If available, a model-specific method for extracting the number of features used by the model is preferable, e.g. counting the number of non-zero weights in a sparse linear regression model. A model-agnostic heuristic is useful when the prediction function is accessible but not the internal structure of the model (e.g. prediction via API call), or when combining preprocessing steps and models complicates programmatic extraction (e.g. training a decision tree on sparse principal components).

The proposed procedure is formally described in Algorithm 1. To estimate whether the j-th feature was used, we sample instances from data , replace their j-th feature values with random values from the distribution of (e.g. by sampling from other instances from ), and observe whether the predictions change. If the prediction of any sample changes, the feature was used for the prediction.

The rate of false positives is zero, i.e. the probability that the heuristic counts a feature as used, but the model did not use the feature is zero. The probability of a false negative, i.e. the heuristic overlooks a feature, depends on the number of samples

, the model function and the data distribution. Let be the probability that the prediction of a random instance depends on the value of . For an instance that depends on for its prediction, let be the probability that a sample from changes the prediction for an instance i. Then the probability of overlooking feature j is: With the simplifying assumption that , the probability that we miss at least one feature is . For a linear model without interactions and only numerical features, the false negative rate is 0: and , so that . Let us assume a non-linear model where only one percent of instances rely on feature () and these instances have a probability of 0.02 that the feature permutation changes the prediction (). If we set , then . If we increase M to 500, the probability that NF counts too few features drops to .

We tested the NF heuristic with the Boston Housing data. We trained decision trees (CART) with maximum depths leading to 1, 2 and 4 features used and LASSO with penalty leading to 0, 2, 3, 4, 11 and 13 features used. For each model we estimated NF with sample sizes and repeated each estimation 100 times. For the elastic net models, NF was always equal to the true number of features. For the CART models the mean absolute differences between NF and the true number of features were 0.280 (), 0.020 () and 0.000 ().

### 3.2 Interaction Strength (IAS)

Interactions between features mean that the prediction cannot be expressed as a sum of independent feature effects, but the effect of a feature depends on values of other features [24]. We propose to measure interaction strength as the scaled approximation error between the ALE main effect model and the prediction function . Based on the ALE decomposition, the ALE main effect model is defined as the sum of first order ALE effects:

 fALE1st(x)=f0+f1,ALE(x1)+…+fp,ALE(xp)

We define interaction strength as the approximation error measured with loss :

 IAS=E(L(f,fALE1st))E(L(f,f0))≥0 (4)

Here, is the mean of the predictions and can be interpreted as the functional decomposition where all feature effects are set to zero. IAS with the loss equals 1 minus the R-squared measure, where the true targets are replaced with :

 IAS=∑ni=1(f(x(i))−fALE1st(x(i)))2∑ni=1(f(x(i))−f0)2=1−R2

If , then , which means that the first order ALE model perfectly approximates and the model has no interactions. IAS can be larger than for additive models for which we would expect , as observed in e.g. Table 1 (e.g. ). This small deviation can occur when the true ALEs (see Equation 2) are not perfectly approximated by finite differences.

### 3.3 Main Effect Complexity (MEC)

To determine the average shape complexity of ALE main effects , we propose the main effect complexity (MEC) measure. For a single ALE main effect, we define as the number of parameters needed to approximate the curve with linear segments. For the entire model, MEC is the average over all main effects, weighted with their variance. Figure 1 shows an ALE plot (= main effect) and its approximation with two linear segments. In the remainder, main effect ALE curves are represented by sparklines, e.g. Figure 1 by 10 0 0 0.0441638219945065 0.232513487915872 0.117035778774142 0.526558195857658 0.171886190181984 0.619642111665782 0.239746475324464 0.657537139241305 0.277414438496296 0.688238693822469 0.317298245186784 0.731395801369906 0.353947207030398 0.765394208820276 0.403829747003651 0.790038933602214 0.469197311658307 0.810493373554827 0.512020956107238 0.834760334926249 0.587409734260159 0.879414206937379 0.645926744526776 0.8919348807605 0.698617576125043 0.900750562309592 0.739096495540082 0.917805223022229 0.770559982404395 0.934469145577491 0.833939564075319 0.955553469032704 0.87392416768727 0.958667634577132 0.896662647836431 0.960851148838697 0.943562558234346 0.973613831586732 1 1 / . Vertical bars, if present, e.g. 10 0.153595113848748 1 0 0 0.0441638219945065 0.232513487915872 0.117035778774142 0.526558195857658 0.171886190181984 0.619642111665782 0.239746475324464 0.657537139241305 0.277414438496296 0.688238693822469 0.317298245186784 0.731395801369906 0.353947207030398 0.765394208820276 0.403829747003651 0.790038933602214 0.469197311658307 0.810493373554827 0.512020956107238 0.834760334926249 0.587409734260159 0.879414206937379 0.645926744526776 0.8919348807605 0.698617576125043 0.900750562309592 0.739096495540082 0.917805223022229 0.770559982404395 0.934469145577491 0.833939564075319 0.955553469032704 0.87392416768727 0.958667634577132 0.896662647836431 0.960851148838697 0.943562558234346 0.973613831586732 1 1 / , show borders between the segments of an approximation.

Through the approximation with linear segments, the degrees of freedom required for describing a main effect curve become measurable. We measure the degrees of freedom as the number of non-zero coefficients for intercepts and slopes of the linear segments. The approximation allows some error, e.g. an almost linear main effect like 10 0 0 0.0132778626776852 0.00285116378037018 0.0295671811436683 0.0170771166425835 0.0444186096679646 0.0431805104607853 0.0658620408890712 0.0215678731796228 0.0849699452768591 0.0215329317714585 0.105231984880058 0.0183626728057456 0.12136451543357 0.0433335848883314 0.134243964732243 0.0781345679052556 0.157303853070161 0.105808925912978 0.177789961325434 0.0955698483393146 0.203828303980362 0.0804713812179251 0.222649318546446 0.163203615903714 0.235881521510398 0.163982111116384 0.259519545670998 0.21312748684833 0.27530289432283 0.214830326044187 0.292155746717243 0.219773138545309 0.310982931817685 0.259072202209324 0.327782065858443 0.268782350336466 0.343556343668399 0.302789759368351 0.358897477692826 0.301012659498059 0.3787467155724 0.3107283300488 0.398263652240928 0.310445689691796 0.417834264319794 0.337406268475739 0.446287338552747 0.373677304069113 0.469529289978424 0.424609539221726 0.482756460405882 0.47768670096473 0.500476617007672 0.488626929485715 0.521995928600725 0.505798558386822 0.539085841400516 0.515389681193714 0.570956622189269 0.552699532240377 0.592526407581769 0.575391094909306 0.608989020063782 0.584587575526664 0.625436861678931 0.649100695582651 0.65369446844157 0.671808034304655 0.680517912462254 0.695693467438897 0.695592232898421 0.738882164624479 0.719814044362073 0.76985727184902 0.739826303469738 0.780908587371888 0.758648272357656 0.800952788086194 0.7787797367416 0.816446604200582 0.804653811411959 0.868723415850169 0.827621203815595 0.891239714015349 0.849122697900249 0.895343079698122 0.868559241181101 0.905973440521492 0.891829338682749 0.927646170908585 0.91171692611012 0.953345090123354 0.937975189419575 0.989660932627234 0.959500886963727 1 0.977364971121896 0.989742338626531 1 0.984347820948623 / may have , even if dozens of parameters would be needed to describe it perfectly. The approximation quality is measured with R-squared, i.e. the proportion of variance of that is explained by the approximation with linear segments. An approximation has to reach an R-squared of at least , where is the user defined maximum approximation error. We also introduced parameter , the maximum number of segments. In the case that an approximation cannot reach an R-squared above with a given , is computed with the maximum number of segments. The selected maximum approximation error should be small, but not too small. We found between and visually meaningful. We apply a post-processing step that greedily sets slopes of the linear segments to zero, as long as R-squared . This post-processing step potentially decreases the , especially for models with constant segments like decision trees or rule-based models. is averaged over all features to obtain the main effect complexity for the model. Each is weighted with the variance of the corresponding ALE main effect to give more weight to features that contribute more to the prediction. For example, the three main effect curves 5 0 0.49921100373065 0.00907467080405462 0.50102392597831 0.0183871457040132 0.50287666233353 0.0256292738785806 0.504300835922156 0.0356844432954238 0.506228091103388 0.0477080178415051 0.50840149377896 0.0587504693065008 0.510206191310913 0.069400952666147 0.511710913279867 0.0777722392442526 0.512694120956889 0.0896216854292794 0.5137349463342 0.100873827571866 0.514295390307273 0.109937417499274 0.514417295151 0.118747449307008 0.514240628964299 0.126843942582414 0.513817160722721 0.135379854003864 0.513101869558373 0.144243227423012 0.512076712571919 0.156461286692254 0.510223672506481 0.165362240711338 0.508584089938172 0.174759083584151 0.506625049735115 0.187801137505781 0.503596284516061 0.199931272523931 0.500555903759373 0.207107235475712 0.498700944901993 0.223095799150128 0.494555313586441 0.235146462716324 0.491545553363525 0.245527842632113 0.489124286945195 0.254880213160131 0.487135054185048 0.26461574262118 0.485303542866825 0.280203018682216 0.482974838459539 0.287451040523817 0.482176763879104 0.298950783297965 0.481313345452969 0.309911250530707 0.480968758103823 0.31825409224015 0.481024774441034 0.332169035447319 0.481722335899825 0.339193169682137 0.482351689348671 0.347028043816302 0.483260706092001 0.355651364960509 0.484495882644824 0.367032102385068 0.486460774470229 0.377135964956108 0.488475567877085 0.385181732119728 0.49022638356728 0.394632393398661 0.492408758380965 0.402540451210935 0.494307486209818 0.412774572837557 0.496815261234992 0.423740929311783 0.499503376459741 0.434051505330361 0.501969756741826 0.441937098122755 0.503780831222911 0.452501499890278 0.506061529043679 0.460693779055701 0.507687009823583 0.472914452892877 0.509832450940047 0.485403877993357 0.51162695564857 0.494070446303977 0.51260864352235 0.502547241005115 0.513344487831131 0.513043529842317 0.513933887609088 0.524018087558145 0.514157359526449 0.536264236021641 0.513925300259344 0.545497938240587 0.513416296948691 0.557692901033943 0.512316201490542 0.56651947461322 0.511228552276207 0.574155352060264 0.510101050138934 0.582667272811947 0.508654357625509 0.59013987590695 0.507234030146715 0.604108639150705 0.504258418507995 0.613238480249725 0.50213351215707 0.624135872457703 0.499469213207509 0.637411871396257 0.496128703060825 0.64784487278233 0.493511221089564 0.657503514846362 0.491157445268549 0.669252604946499 0.488463919396734 0.676796072265938 0.486874380885034 0.685491334440598 0.485215345299615 0.693579436175783 0.483867585271647 0.700547641949166 0.482877001987978 0.71049974847542 0.481764063808388 0.718930966074907 0.481119484722152 0.731563828817977 0.480695364398809 0.74092146202287 0.480811388662037 0.751781085195682 0.481404891540792 0.760212544569914 0.482195965173528 0.775503861789395 0.484313244604574 0.786919258019664 0.486398994675349 0.796010127731655 0.48831749334494 0.802930834317453 0.489903017154903 0.814154570023779 0.492645718454323 0.822467048738328 0.494765572985638 0.83547499588329 0.498129526873955 0.847586896939602 0.501194971191777 0.859670956166572 0.504069345521028 0.871150267111353 0.506535252597626 0.879213938416926 0.508071861772251 0.888384497830326 0.509594464809613 0.894470081370667 0.510463113376834 0.902916842430947 0.511475206902742 0.91203710547669 0.512317832062143 0.920475130269753 0.512876702103195 0.93114915172852 0.513305738056481 0.939500028139156 0.513449853375996 0.948260958011693 0.513448514794428 0.956124363662118 0.513339328086252 0.964237510512418 0.513144858543175 0.977810524601265 0.512698205193198 0.986652387754562 0.512364673771948 1 0.511843501051419 / , 5 0 0.492110049406885 0.00907467080405462 0.510239269792169 0.0183871457040132 0.528766631204992 0.0256292738785806 0.543008365442004 0.0356844432954238 0.562280915007892 0.0477080178415051 0.584014939190431 0.0587504693065008 0.602061912310836 0.069400952666147 0.617109130080688 0.0777722392442526 0.626941205511677 0.0896216854292794 0.63734945768623 0.100873827571866 0.642953896259774 0.109937417499274 0.644172944041531 0.118747449307008 0.642406281784024 0.126843942582414 0.638171599226643 0.135379854003864 0.631018687656421 0.144243227423012 0.620767118100303 0.156461286692254 0.602236718229615 0.165362240711338 0.585840893350693 0.174759083584151 0.566250492349739 0.187801137505781 0.535962841827203 0.199931272523931 0.505559035975228 0.207107235475712 0.487009448452092 0.223095799150128 0.445553137621545 0.235146462716324 0.415455537030231 0.245527842632113 0.391242874108898 0.254880213160131 0.371350547486945 0.26461574262118 0.35303543513775 0.280203018682216 0.329748391955305 0.287451040523817 0.321767646367501 0.298950783297965 0.313133462188789 0.309911250530707 0.309687588487701 0.31825409224015 0.310247751524509 0.332169035447319 0.317223365257703 0.339193169682137 0.32351689919663 0.347028043816302 0.332607065943535 0.355651364960509 0.344958830648823 0.367032102385068 0.364607747752274 0.377135964956108 0.384755680782383 0.385181732119728 0.402263836873833 0.394632393398661 0.424087584103569 0.402540451210935 0.443074861686667 0.412774572837557 0.468152611116003 0.423740929311783 0.495033762611151 0.434051505330361 0.519697564853832 0.441937098122755 0.537808309306148 0.452501499890278 0.560615287139167 0.460693779055701 0.57687009472115 0.472914452892877 0.598324505657927 0.485403877993357 0.616269552595331 0.494070446303977 0.626086431261033 0.502547241005115 0.633444874289953 0.513043529842317 0.639338872000424 0.524018087558145 0.641573591096695 0.536264236021641 0.639252998330179 0.545497938240587 0.634162965151278 0.557692901033943 0.623162010493386 0.56651947461322 0.612285518325635 0.574155352060264 0.601010496966005 0.582667272811947 0.586543571898212 0.59013987590695 0.572340297226093 0.604108639150705 0.542584181236831 0.613238480249725 0.521335118138844 0.624135872457703 0.494692129309804 0.637411871396257 0.461287028921356 0.64784487278233 0.43511221025036 0.657503514846362 0.411574453135954 0.669252604946499 0.384639195878973 0.676796072265938 0.368743811746509 0.685491334440598 0.352153457040172 0.693579436175783 0.338675857808932 0.700547641949166 0.328770025834349 0.71049974847542 0.317640645157596 0.718930966074907 0.311194855097838 0.731563828817977 0.306953652735017 0.74092146202287 0.308113895702164 0.751781085195682 0.314048924500567 0.760212544569914 0.321959660538736 0.775503861789395 0.343132453656374 0.786919258019664 0.36398995294637 0.796010127731655 0.383174938229278 0.802930834317453 0.399030175112117 0.814154570023779 0.426457185941937 0.822467048738328 0.447655729561035 0.83547499588329 0.481295265783012 0.847586896939602 0.511949706648998 0.859670956166572 0.540693447981941 0.871150267111353 0.565352517362868 0.879213938416926 0.58071860848407 0.888384497830326 0.595944638546232 0.894470081370667 0.604631124263811 0.902916842430947 0.614752059931057 0.91203710547669 0.623178312413501 0.920475130269753 0.6287670140427 0.93114915172852 0.633057375619024 0.939500028139156 0.634498530760325 0.948260958011693 0.634485147264251 0.956124363662118 0.633393282461851 0.964237510512418 0.631448589532825 0.977810524601265 0.626982060441331 0.986652387754562 0.623646749178878 1 0.618435026459563 / and 5 0 0.476331993495636 0.00907467080405462 0.530719335731934 0.0183871457040132 0.586301093720036 0.0256292738785806 0.629026044926315 0.0356844432954238 0.686843351048819 0.0477080178415051 0.75204503119394 0.0587504693065008 0.806185615194782 0.069400952666147 0.851326975757926 0.0777722392442526 0.880822997821449 0.0896216854292794 0.912047510570302 0.100873827571866 0.928860649823101 0.109937417499274 0.932517693204879 0.118747449307008 0.927217646882687 0.126843942582414 0.91451357761581 0.135379854003864 0.893054854077674 0.144243227423012 0.862300192441638 0.156461286692254 0.80670911234101 0.165362240711338 0.757521760338184 0.174759083584151 0.698750714347822 0.187801137505781 0.60788801714672 0.199931272523931 0.516676861108563 0.207107235475712 0.461028258762111 0.223095799150128 0.336659680821013 0.235146462716324 0.246367128814723 0.245527842632113 0.173729332496948 0.254880213160131 0.114052502003557 0.26461574262118 0.0591072919925342 0.280203018682216 -0.0107537017685309 0.287451040523817 -0.0346959055102422 0.298950783297965 -0.0605984454437355 0.309911250530707 -0.0709360985135132 0.31825409224015 -0.0692556605346054 0.332169035447319 -0.0483289496769449 0.339193169682137 -0.0294484316606739 0.347028043816302 -0.00217803609396502 0.355651364960509 0.0348771325272514 0.367032102385068 0.0938237083744217 0.377135964956108 0.154267349104612 0.385181732119728 0.2067916937807 0.394632393398661 0.272262797136777 0.402540451210935 0.329224522309464 0.412774572837557 0.404457645183431 0.423740929311783 0.485100984937194 0.434051505330361 0.559092303495742 0.441937098122755 0.613424482175909 0.452501499890278 0.681845358539866 0.460693779055701 0.730609748183052 0.472914452892877 0.79497294624369 0.485403877993357 0.848808064511091 0.494070446303977 0.878258689512505 0.502547241005115 0.90033400961793 0.513043529842317 0.918015992211501 0.524018087558145 0.924720137705726 0.536264236021641 0.917758344848005 0.545497938240587 0.902488234275372 0.557692901033943 0.869485358651699 0.56651947461322 0.836855878429149 0.574155352060264 0.803030816349307 0.582667272811947 0.759630051281862 0.59013987590695 0.717020244928053 0.604108639150705 0.627751957646771 0.613238480249725 0.564004831070862 0.624135872457703 0.484075966236211 0.637411871396257 0.38386082952292 0.64784487278233 0.305336532353953 0.657503514846362 0.234723428109404 0.669252604946499 0.153917879162665 0.676796072265938 0.106231876903044 0.685491334440598 0.0564609878270583 0.693579436175783 0.0160283500179453 0.700547641949166 -0.0136890144389348 0.71049974847542 -0.0470769858036873 0.718930966074907 -0.0664142335905017 0.731563828817977 -0.0791377079158902 0.74092146202287 -0.0756569279490937 0.751781085195682 -0.0578518398997277 0.760212544569914 -0.034119675887669 0.775503861789395 0.0293985215621611 0.786919258019664 0.0919708032287782 0.796010127731655 0.149525543598704 0.802930834317453 0.197091068690397 0.814154570023779 0.279371771120187 0.822467048738328 0.34296714364064 0.83547499588329 0.443885346484776 0.847586896939602 0.535848316475688 0.859670956166572 0.622079241648019 0.871150267111353 0.696056238575 0.879213938416926 0.742154416621028 0.888384497830326 0.787832459312858 0.894470081370667 0.813891923384387 0.902916842430947 0.844254792630657 0.91203710547669 0.869533685560174 0.920475130269753 0.886299976292437 0.93114915172852 0.899171372642076 0.939500028139156 0.903495134847559 0.948260958011693 0.903455338090458 0.956124363662118 0.900180091277322 0.964237510512418 0.894346393997881 0.977810524601265 0.88094747896879 0.986652387754562 0.870941995053 1 0.85530751098978 / differ from each other by a scaling factor in the y-axis, but each requires five segments to be approximated with : 5 0.095281314742052 1 0.326256148630134 1 0.526069636642834 1 0.715591029158772 1 0 0.49921100373065 0.00907467080405462 0.50102392597831 0.0183871457040132 0.50287666233353 0.0256292738785806 0.504300835922156 0.0356844432954238 0.506228091103388 0.0477080178415051 0.50840149377896 0.0587504693065008 0.510206191310913 0.069400952666147 0.511710913279867 0.0777722392442526 0.512694120956889 0.0896216854292794 0.5137349463342 0.100873827571866 0.514295390307273 0.109937417499274 0.514417295151 0.118747449307008 0.514240628964299 0.126843942582414 0.513817160722721 0.135379854003864 0.513101869558373 0.144243227423012 0.512076712571919 0.156461286692254 0.510223672506481 0.165362240711338 0.508584089938172 0.174759083584151 0.506625049735115 0.187801137505781 0.503596284516061 0.199931272523931 0.500555903759373 0.207107235475712 0.498700944901993 0.223095799150128 0.494555313586441 0.235146462716324 0.491545553363525 0.245527842632113 0.489124286945195 0.254880213160131 0.487135054185048 0.26461574262118 0.485303542866825 0.280203018682216 0.482974838459539 0.287451040523817 0.482176763879104 0.298950783297965 0.481313345452969 0.309911250530707 0.480968758103823 0.31825409224015 0.481024774441034 0.332169035447319 0.481722335899825 0.339193169682137 0.482351689348671 0.347028043816302 0.483260706092001 0.355651364960509 0.484495882644824 0.367032102385068 0.486460774470229 0.377135964956108 0.488475567877085 0.385181732119728 0.49022638356728 0.394632393398661 0.492408758380965 0.402540451210935 0.494307486209818 0.412774572837557 0.496815261234992 0.423740929311783 0.499503376459741 0.434051505330361 0.501969756741826 0.441937098122755 0.503780831222911 0.452501499890278 0.506061529043679 0.460693779055701 0.507687009823583 0.472914452892877 0.509832450940047 0.485403877993357 0.51162695564857 0.494070446303977 0.51260864352235 0.502547241005115 0.513344487831131 0.513043529842317 0.513933887609088 0.524018087558145 0.514157359526449 0.536264236021641 0.513925300259344 0.545497938240587 0.513416296948691 0.557692901033943 0.512316201490542 0.56651947461322 0.511228552276207 0.574155352060264 0.510101050138934 0.582667272811947 0.508654357625509 0.59013987590695 0.507234030146715 0.604108639150705 0.504258418507995 0.613238480249725 0.50213351215707 0.624135872457703 0.499469213207509 0.637411871396257 0.496128703060825 0.64784487278233 0.493511221089564 0.657503514846362 0.491157445268549 0.669252604946499 0.488463919396734 0.676796072265938 0.486874380885034 0.685491334440598 0.485215345299615 0.693579436175783 0.483867585271647 0.700547641949166 0.482877001987978 0.71049974847542 0.481764063808388 0.718930966074907 0.481119484722152 0.731563828817977 0.480695364398809 0.74092146202287 0.480811388662037 0.751781085195682 0.481404891540792 0.760212544569914 0.482195965173528 0.775503861789395 0.484313244604574 0.786919258019664 0.486398994675349 0.796010127731655 0.48831749334494 0.802930834317453 0.489903017154903 0.814154570023779 0.492645718454323 0.822467048738328 0.494765572985638 0.83547499588329 0.498129526873955 0.847586896939602 0.501194971191777 0.859670956166572 0.504069345521028 0.871150267111353 0.506535252597626 0.879213938416926 0.508071861772251 0.888384497830326 0.509594464809613 0.894470081370667 0.510463113376834 0.902916842430947 0.511475206902742 0.91203710547669 0.512317832062143 0.920475130269753 0.512876702103195 0.93114915172852 0.513305738056481 0.939500028139156 0.513449853375996 0.948260958011693 0.513448514794428 0.956124363662118 0.513339328086252 0.964237510512418 0.513144858543175 0.977810524601265 0.512698205193198 0.986652387754562 0.512364673771948 1 0.511843501051419 / , 5 0.095281314742052 1 0.326256148630134 1 0.526069636642834 1 0.715591029833964 1 0 0.492110049406885 0.00907467080405462 0.510239269792169 0.0183871457040132 0.528766631204992 0.0256292738785806 0.543008365442004 0.0356844432954238 0.562280915007892 0.0477080178415051 0.584014939190431 0.0587504693065008 0.602061912310836 0.069400952666147 0.617109130080688 0.0777722392442526 0.626941205511677 0.0896216854292794 0.63734945768623 0.100873827571866 0.642953896259774 0.109937417499274 0.644172944041531 0.118747449307008 0.642406281784024 0.126843942582414 0.638171599226643 0.135379854003864 0.631018687656421 0.144243227423012 0.620767118100303 0.156461286692254 0.602236718229615 0.165362240711338 0.585840893350693 0.174759083584151 0.566250492349739 0.187801137505781 0.535962841827203 0.199931272523931 0.505559035975228 0.207107235475712 0.487009448452092 0.223095799150128 0.445553137621545 0.235146462716324 0.415455537030231 0.245527842632113 0.391242874108898 0.254880213160131 0.371350547486945 0.26461574262118 0.35303543513775 0.280203018682216 0.329748391955305 0.287451040523817 0.321767646367501 0.298950783297965 0.313133462188789 0.309911250530707 0.309687588487701 0.31825409224015 0.310247751524509 0.332169035447319 0.317223365257703 0.339193169682137 0.32351689919663 0.347028043816302 0.332607065943535 0.355651364960509 0.344958830648823 0.367032102385068 0.364607747752274 0.377135964956108 0.384755680782383 0.385181732119728 0.402263836873833 0.394632393398661 0.424087584103569 0.402540451210935 0.443074861686667 0.412774572837557 0.468152611116003 0.423740929311783 0.495033762611151 0.434051505330361 0.519697564853832 0.441937098122755 0.537808309306148 0.452501499890278 0.560615287139167 0.460693779055701 0.57687009472115 0.472914452892877 0.598324505657927 0.485403877993357 0.616269552595331 0.494070446303977 0.626086431261033 0.502547241005115 0.633444874289953 0.513043529842317 0.639338872000424 0.524018087558145 0.641573591096695 0.536264236021641 0.639252998330179 0.545497938240587 0.634162965151278 0.557692901033943 0.623162010493386 0.56651947461322 0.612285518325635 0.574155352060264 0.601010496966005 0.582667272811947 0.586543571898212 0.59013987590695 0.572340297226093 0.604108639150705 0.542584181236831 0.613238480249725 0.521335118138844 0.624135872457703 0.494692129309804 0.637411871396257 0.461287028921356 0.64784487278233 0.43511221025036 0.657503514846362 0.411574453135954 0.669252604946499 0.384639195878973 0.676796072265938 0.368743811746509 0.685491334440598 0.352153457040172 0.693579436175783 0.338675857808932 0.700547641949166 0.328770025834349 0.71049974847542 0.317640645157596 0.718930966074907 0.311194855097838 0.731563828817977 0.306953652735017 0.74092146202287 0.308113895702164 0.751781085195682 0.314048924500567 0.760212544569914 0.321959660538736 0.775503861789395 0.343132453656374 0.786919258019664 0.36398995294637 0.796010127731655 0.383174938229278 0.802930834317453 0.399030175112117 0.814154570023779 0.426457185941937 0.822467048738328 0.447655729561035 0.83547499588329 0.481295265783012 0.847586896939602 0.511949706648998 0.859670956166572 0.540693447981941 0.871150267111353 0.565352517362868 0.879213938416926 0.58071860848407 0.888384497830326 0.595944638546232 0.894470081370667 0.604631124263811 0.902916842430947 0.614752059931057 0.91203710547669 0.623178312413501 0.920475130269753 0.6287670140427 0.93114915172852 0.633057375619024 0.939500028139156 0.634498530760325 0.948260958011693 0.634485147264251 0.956124363662118 0.633393282461851 0.964237510512418 0.631448589532825 0.977810524601265 0.626982060441331 0.986652387754562 0.623646749178878 1 0.618435026459563 / and 5 0.095281314742052 1 0.326256148630134 1 0.526069636642834 1 0.715591064154931 1 0 0.476331993495636 0.00907467080405462 0.530719335731934 0.0183871457040132 0.586301093720036 0.0256292738785806 0.629026044926315 0.0356844432954238 0.686843351048819 0.0477080178415051 0.75204503119394 0.0587504693065008 0.806185615194782 0.069400952666147 0.851326975757926 0.0777722392442526 0.880822997821449 0.0896216854292794 0.912047510570302 0.100873827571866 0.928860649823101 0.109937417499274 0.932517693204879 0.118747449307008 0.927217646882687 0.126843942582414 0.91451357761581 0.135379854003864 0.893054854077674 0.144243227423012 0.862300192441638 0.156461286692254 0.80670911234101 0.165362240711338 0.757521760338184 0.174759083584151 0.698750714347822 0.187801137505781 0.60788801714672 0.199931272523931 0.516676861108563 0.207107235475712 0.461028258762111 0.223095799150128 0.336659680821013 0.235146462716324 0.246367128814723 0.245527842632113 0.173729332496948 0.254880213160131 0.114052502003557 0.26461574262118 0.0591072919925342 0.280203018682216 -0.0107537017685309 0.287451040523817 -0.0346959055102422 0.298950783297965 -0.0605984454437355 0.309911250530707 -0.0709360985135132 0.31825409224015 -0.0692556605346054 0.332169035447319 -0.0483289496769449 0.339193169682137 -0.0294484316606739 0.347028043816302 -0.00217803609396502 0.355651364960509 0.0348771325272514 0.367032102385068 0.0938237083744217 0.377135964956108 0.154267349104612 0.385181732119728 0.2067916937807 0.394632393398661 0.272262797136777 0.402540451210935 0.329224522309464 0.412774572837557 0.404457645183431 0.423740929311783 0.485100984937194 0.434051505330361 0.559092303495742 0.441937098122755 0.613424482175909 0.452501499890278 0.681845358539866 0.460693779055701 0.730609748183052 0.472914452892877 0.79497294624369 0.485403877993357 0.848808064511091 0.494070446303977 0.878258689512505 0.502547241005115 0.90033400961793 0.513043529842317 0.918015992211501 0.524018087558145 0.924720137705726 0.536264236021641 0.917758344848005 0.545497938240587 0.902488234275372 0.557692901033943 0.869485358651699 0.56651947461322 0.836855878429149 0.574155352060264 0.803030816349307 0.582667272811947 0.759630051281862 0.59013987590695 0.717020244928053 0.604108639150705 0.627751957646771 0.613238480249725 0.564004831070862 0.624135872457703 0.484075966236211 0.637411871396257 0.38386082952292 0.64784487278233 0.305336532353953 0.657503514846362 0.234723428109404 0.669252604946499 0.153917879162665 0.676796072265938 0.106231876903044 0.685491334440598 0.0564609878270583 0.693579436175783 0.0160283500179453 0.700547641949166 -0.0136890144389348 0.71049974847542 -0.0470769858036873 0.718930966074907 -0.0664142335905017 0.731563828817977 -0.0791377079158902 0.74092146202287 -0.0756569279490937 0.751781085195682 -0.0578518398997277 0.760212544569914 -0.034119675887669 0.775503861789395 0.0293985215621611 0.786919258019664 0.0919708032287782 0.796010127731655 0.149525543598704 0.802930834317453 0.197091068690397 0.814154570023779 0.279371771120187 0.822467048738328 0.34296714364064 0.83547499588329 0.443885346484776 0.847586896939602 0.535848316475688 0.859670956166572 0.622079241648019 0.871150267111353 0.696056238575 0.879213938416926 0.742154416621028 0.888384497830326 0.787832459312858 0.894470081370667 0.813891923384387 0.902916842430947 0.844254792630657 0.91203710547669 0.869533685560174 0.920475130269753 0.886299976292437 0.93114915172852 0.899171372642076 0.939500028139156 0.903495134847559 0.948260958011693 0.903455338090458 0.956124363662118 0.900180091277322 0.964237510512418 0.894346393997881 0.977810524601265 0.88094747896879 0.986652387754562 0.870941995053 1 0.85530751098978 / . When weighted with the variance, 5 0 0.49921100373065 0.00907467080405462 0.50102392597831 0.0183871457040132 0.50287666233353 0.0256292738785806 0.504300835922156 0.0356844432954238 0.506228091103388 0.0477080178415051 0.50840149377896 0.0587504693065008 0.510206191310913 0.069400952666147 0.511710913279867 0.0777722392442526 0.512694120956889 0.0896216854292794 0.5137349463342 0.100873827571866 0.514295390307273 0.109937417499274 0.514417295151 0.118747449307008 0.514240628964299 0.126843942582414 0.513817160722721 0.135379854003864 0.513101869558373 0.144243227423012 0.512076712571919 0.156461286692254 0.510223672506481 0.165362240711338 0.508584089938172 0.174759083584151 0.506625049735115 0.187801137505781 0.503596284516061 0.199931272523931 0.500555903759373 0.207107235475712 0.498700944901993 0.223095799150128 0.494555313586441 0.235146462716324 0.491545553363525 0.245527842632113 0.489124286945195 0.254880213160131 0.487135054185048 0.26461574262118 0.485303542866825 0.280203018682216 0.482974838459539 0.287451040523817 0.482176763879104 0.298950783297965 0.481313345452969 0.309911250530707 0.480968758103823 0.31825409224015 0.481024774441034 0.332169035447319 0.481722335899825 0.339193169682137 0.482351689348671 0.347028043816302 0.483260706092001 0.355651364960509 0.484495882644824 0.367032102385068 0.486460774470229 0.377135964956108 0.488475567877085 0.385181732119728 0.49022638356728 0.394632393398661 0.492408758380965 0.402540451210935 0.494307486209818 0.412774572837557 0.496815261234992 0.423740929311783 0.499503376459741 0.434051505330361 0.501969756741826 0.441937098122755 0.503780831222911 0.452501499890278 0.506061529043679 0.460693779055701 0.507687009823583 0.472914452892877 0.509832450940047 0.485403877993357 0.51162695564857 0.494070446303977 0.51260864352235 0.502547241005115 0.513344487831131 0.513043529842317 0.513933887609088 0.524018087558145 0.514157359526449 0.536264236021641 0.513925300259344 0.545497938240587 0.513416296948691 0.557692901033943 0.512316201490542 0.56651947461322 0.511228552276207 0.574155352060264 0.510101050138934 0.582667272811947 0.508654357625509 0.59013987590695 0.507234030146715 0.604108639150705 0.504258418507995 0.613238480249725 0.50213351215707 0.624135872457703 0.499469213207509 0.637411871396257 0.496128703060825 0.64784487278233 0.493511221089564 0.657503514846362 0.491157445268549 0.669252604946499 0.488463919396734 0.676796072265938 0.486874380885034 0.685491334440598 0.485215345299615 0.693579436175783 0.483867585271647 0.700547641949166 0.482877001987978 0.71049974847542 0.481764063808388 0.718930966074907 0.481119484722152 0.731563828817977 0.480695364398809 0.74092146202287 0.480811388662037 0.751781085195682 0.481404891540792 0.760212544569914 0.482195965173528 0.775503861789395 0.484313244604574 0.786919258019664 0.486398994675349 0.796010127731655 0.48831749334494 0.802930834317453 0.489903017154903 0.814154570023779 0.492645718454323 0.822467048738328 0.494765572985638 0.83547499588329 0.498129526873955 0.847586896939602 0.501194971191777 0.859670956166572 0.504069345521028 0.871150267111353 0.506535252597626 0.879213938416926 0.508071861772251 0.888384497830326 0.509594464809613 0.894470081370667 0.510463113376834 0.902916842430947 0.511475206902742 0.91203710547669 0.512317832062143 0.920475130269753 0.512876702103195 0.93114915172852 0.513305738056481 0.939500028139156 0.513449853375996 0.948260958011693 0.513448514794428 0.956124363662118 0.513339328086252 0.964237510512418 0.513144858543175 0.977810524601265 0.512698205193198 0.986652387754562 0.512364673771948 1 0.511843501051419 / gets a weight of 0.02, 5 0 0.492110049406885 0.00907467080405462 0.510239269792169 0.0183871457040132 0.528766631204992 0.0256292738785806 0.543008365442004 0.0356844432954238 0.562280915007892 0.0477080178415051 0.584014939190431 0.0587504693065008 0.602061912310836 0.069400952666147 0.617109130080688 0.0777722392442526 0.626941205511677 0.0896216854292794 0.63734945768623 0.100873827571866 0.642953896259774 0.109937417499274 0.644172944041531 0.118747449307008 0.642406281784024 0.126843942582414 0.638171599226643 0.135379854003864 0.631018687656421 0.144243227423012 0.620767118100303 0.156461286692254 0.602236718229615 0.165362240711338 0.585840893350693 0.174759083584151 0.566250492349739 0.187801137505781 0.535962841827203 0.199931272523931 0.505559035975228 0.207107235475712 0.487009448452092 0.223095799150128 0.445553137621545 0.235146462716324 0.415455537030231 0.245527842632113 0.391242874108898 0.254880213160131 0.371350547486945 0.26461574262118 0.35303543513775 0.280203018682216 0.329748391955305 0.287451040523817 0.321767646367501 0.298950783297965 0.313133462188789 0.309911250530707 0.309687588487701 0.31825409224015 0.310247751524509 0.332169035447319 0.317223365257703 0.339193169682137 0.32351689919663 0.347028043816302 0.332607065943535 0.355651364960509 0.344958830648823 0.367032102385068 0.364607747752274 0.377135964956108 0.384755680782383 0.385181732119728 0.402263836873833 0.394632393398661 0.424087584103569 0.402540451210935 0.443074861686667 0.412774572837557 0.468152611116003 0.423740929311783 0.495033762611151 0.434051505330361 0.519697564853832 0.441937098122755 0.537808309306148 0.452501499890278 0.560615287139167 0.460693779055701 0.57687009472115 0.472914452892877 0.598324505657927 0.485403877993357 0.616269552595331 0.494070446303977 0.626086431261033 0.502547241005115 0.633444874289953 0.513043529842317 0.639338872000424 0.524018087558145 0.641573591096695 0.536264236021641 0.639252998330179 0.545497938240587 0.634162965151278 0.557692901033943 0.623162010493386 0.56651947461322 0.612285518325635 0.574155352060264 0.601010496966005 0.582667272811947 0.586543571898212 0.59013987590695 0.572340297226093 0.604108639150705 0.542584181236831 0.613238480249725 0.521335118138844 0.624135872457703 0.494692129309804 0.637411871396257 0.461287028921356 0.64784487278233 0.43511221025036 0.657503514846362 0.411574453135954 0.669252604946499 0.384639195878973 0.676796072265938 0.368743811746509 0.685491334440598 0.352153457040172 0.693579436175783 0.338675857808932 0.700547641949166 0.328770025834349 0.71049974847542 0.317640645157596 0.718930966074907 0.311194855097838 0.731563828817977 0.306953652735017 0.74092146202287 0.308113895702164 0.751781085195682 0.314048924500567 0.760212544569914 0.321959660538736 0.775503861789395 0.343132453656374 0.786919258019664 0.36398995294637 0.796010127731655 0.383174938229278 0.802930834317453 0.399030175112117 0.814154570023779 0.426457185941937 0.822467048738328 0.447655729561035 0.83547499588329 0.481295265783012 0.847586896939602 0.511949706648998 0.859670956166572 0.540693447981941 0.871150267111353 0.565352517362868 0.879213938416926 0.58071860848407 0.888384497830326 0.595944638546232 0.894470081370667 0.604631124263811 0.902916842430947 0.614752059931057 0.91203710547669 0.623178312413501 0.920475130269753 0.6287670140427 0.93114915172852 0.633057375619024 0.939500028139156 0.634498530760325 0.948260958011693 0.634485147264251 0.956124363662118 0.633393282461851 0.964237510512418 0.631448589532825 0.977810524601265 0.626982060441331 0.986652387754562 0.623646749178878 1 0.618435026459563 / a weight of 2.03 and 5 0 0.476331993495636 0.00907467080405462 0.530719335731934 0.0183871457040132 0.586301093720036 0.0256292738785806 0.629026044926315 0.0356844432954238 0.686843351048819 0.0477080178415051 0.75204503119394 0.0587504693065008 0.806185615194782 0.069400952666147 0.851326975757926 0.0777722392442526 0.880822997821449 0.0896216854292794 0.912047510570302 0.100873827571866 0.928860649823101 0.109937417499274 0.932517693204879 0.118747449307008 0.927217646882687 0.126843942582414 0.91451357761581 0.135379854003864 0.893054854077674 0.144243227423012 0.862300192441638 0.156461286692254 0.80670911234101 0.165362240711338 0.757521760338184 0.174759083584151 0.698750714347822 0.187801137505781 0.60788801714672 0.199931272523931 0.516676861108563 0.207107235475712 0.461028258762111 0.223095799150128 0.336659680821013 0.235146462716324 0.246367128814723 0.245527842632113 0.173729332496948 0.254880213160131 0.114052502003557 0.26461574262118 0.0591072919925342 0.280203018682216 -0.0107537017685309 0.287451040523817 -0.0346959055102422 0.298950783297965 -0.0605984454437355 0.309911250530707 -0.0709360985135132 0.31825409224015 -0.0692556605346054 0.332169035447319 -0.0483289496769449 0.339193169682137 -0.0294484316606739 0.347028043816302 -0.00217803609396502 0.355651364960509 0.0348771325272514 0.367032102385068 0.0938237083744217 0.377135964956108 0.154267349104612 0.385181732119728 0.2067916937807 0.394632393398661 0.272262797136777 0.402540451210935 0.329224522309464 0.412774572837557 0.404457645183431 0.423740929311783 0.485100984937194 0.434051505330361 0.559092303495742 0.441937098122755 0.613424482175909 0.452501499890278 0.681845358539866 0.460693779055701 0.730609748183052 0.472914452892877 0.79497294624369 0.485403877993357 0.848808064511091 0.494070446303977 0.878258689512505 0.502547241005115 0.90033400961793 0.513043529842317 0.918015992211501 0.524018087558145 0.924720137705726 0.536264236021641 0.917758344848005 0.545497938240587 0.902488234275372 0.557692901033943 0.869485358651699 0.56651947461322 0.836855878429149 0.574155352060264 0.803030816349307 0.582667272811947 0.759630051281862 0.59013987590695 0.717020244928053 0.604108639150705 0.627751957646771 0.613238480249725 0.564004831070862 0.624135872457703 0.484075966236211 0.637411871396257 0.38386082952292 0.64784487278233 0.305336532353953 0.657503514846362 0.234723428109404 0.669252604946499 0.153917879162665 0.676796072265938 0.106231876903044 0.685491334440598 0.0564609878270583 0.693579436175783 0.0160283500179453 0.700547641949166 -0.0136890144389348 0.71049974847542 -0.0470769858036873 0.718930966074907 -0.0664142335905017 0.731563828817977 -0.0791377079158902 0.74092146202287 -0.0756569279490937 0.751781085195682 -0.0578518398997277 0.760212544569914 -0.034119675887669 0.775503861789395 0.0293985215621611 0.786919258019664 0.0919708032287782 0.796010127731655 0.149525543598704 0.802930834317453 0.197091068690397 0.814154570023779 0.279371771120187 0.822467048738328 0.34296714364064 0.83547499588329 0.443885346484776 0.847586896939602 0.535848316475688 0.859670956166572 0.622079241648019 0.871150267111353 0.696056238575 0.879213938416926 0.742154416621028 0.888384497830326 0.787832459312858 0.894470081370667 0.813891923384387 0.902916842430947 0.844254792630657 0.91203710547669 0.869533685560174 0.920475130269753 0.886299976292437 0.93114915172852 0.899171372642076 0.939500028139156 0.903495134847559 0.948260958011693 0.903455338090458 0.956124363662118 0.900180091277322 0.964237510512418 0.894346393997881 0.977810524601265 0.88094747896879 0.986652387754562 0.870941995053 1 0.85530751098978 / a weight of 18.28. Algorithm 2 describes the MEC computation in detail.

## 4 Improving Post-hoc Interpretation

Minimizing the number of features (NF), the interaction strength (IAS) and the main effect complexity (AMEC) improves reliability and compactness of post-hoc interpretation methods such as partial dependence plots, ALE plots, feature importance, interaction effects and local surrogate models.

#### 4.0.1 The less features, the less verbose the interpretations.

Our NF measure improves the readability of post-hoc analysis results. The computational complexity and output size of most interpretation methods scales with , like feature effect plots [3, 16] or feature importance [12, 8]. As shown in Table 2, a model with fewer features has a more compact representation and if additionally , the ALE main effects fully characterize the prediction function. Interpretation methods that analyze 2-way feature interactions scale with . A complete functional decomposition [18, 3] would require to estimate components which has a computational complexity of .

#### 4.0.2 The less interaction, the more reliable feature effects.

Feature effect plots, such as partial dependence plots and ALE plots visualize the marginal relationship between a feature and the prediction. The estimated effects are averages across instances. The effects can vary greatly for individual instances and even have opposite directions when the model includes feature interactions.

In the following simulation, we trained three models with different capabilities of modeling interactions between features: a linear regression model, a support vector machine (radial basis kernel, C=0.05) and gradient boosted trees. We simulated 500 data points with 4 features and a continuous target based on Friedman et. al (1991)

[17]

. The features are uniformly distributed in the following intervals:

, , , . The target was simulated as: , where . Figure 2 shows an increasing interaction strength depending on the model used. This means that we prefer models with less interaction when using feature effect plots.

#### 4.0.3 The less complex the main effects, the better summarizable.

In linear models, a feature effect can be expressed by a single number, the regression coefficient. If effects are non-linear the method of choice is visualization [3, 16]. Summarizing the effects with a single number (e.g. average marginal effects [22]) can be misleading, e.g. if the effect has a U-shape, the average effect might be zero. As a by-product of MEC, there is a third option: Instead of reporting a single number, the coefficients of the segmented linear model can be reported. Minimizing MEC means preferring models with main effects that can be described with fewer numbers, offering a more compact model description.

## 5 Optimization of Performance and Interpretability

As one of the main applications of the proposed interpretability measures, we demonstrate model selection for performance and interpretability in a multi-objective optimization approach.

#### 5.0.1 Predicting Wine Quality.

We used the wine quality dataset [9] which contains physical-chemical properties such as alcohol and residual sugar of 4870 white wines. The goal was to predict wine quality on a scale of 0 to 10, assessed by the median of three blind ratings.

#### 5.0.2 Motivation.

As [14] emphasizes, it is difficult to know the desired compromise between interpretability and performance before modeling the data and suggests multi-objective optimization. This stands in contrast to a priori selecting an interpretable model class (e.g. decision rules) and optimizing within this class or exclusively optimizing performance and applying post-hoc interpretations. We suggest searching over a wide spectrum of model classes and hyper parameter settings, presenting the set of Pareto optimal models, and allowing the practitioner to choose a suitable compromise between interpretability and performance. The three measures of interpretability provide a detailed characterization of machine learning models, which enables making informed decisions (e.g. how much does performance suffer if we use a model without interactions?).

#### 5.0.3 Optimization Setup.

We used the mlrMBO model-based optimization framework [19] to find the best model based on four objectives: number of features used by the model (NF), main effect complexity (MEC), interaction strength (IAS) and the cross-validated mean absolute error (MAE). We optimized over the space of following model classes (and hyperparameters): CART (maximum tree-depth and pruning cp), support vector machine (cost C and sigma), elastic net regression (regularization alpha and penalization lambda), gradient boosted trees (maximum depth, number of iterations), gradient boosted generalized additive model (number of iterations) and random forest (mtry).

#### 5.0.4 Model-based Optimization Setup.

We used the ParEGO algorithm [21] for multi-objective optimization. Within the fitness function, the MAE was estimated using 5-fold cross-validation and the other measures (NF, MEC, IAS) were estimated using all data instances. We set the number of iterations of ParEGO to 350. For all other parameters of the model-based, multi-objective optimization, we relied on the sensitive defaults provided by [7].

#### 5.0.5 Results.

Table 1 shows the set of Pareto-optimal models along with their MAE, NF, IAS and MEC. If two models from the same class with different hyperparameter values had exactly the same MAE, NF, IAS and MEC, we kept only one and dropped the others. We also dropped constant models (e.g. elastic net regression with strong penalization), with exception of the median model. For a more informative visualization, we propose to visualize the main effects together with the measures in Table 2. The four selected models show different trade-offs between the four measures.

The complexity measures allow to study the trade-off between interpretability and performance across different model classes and hyperparameter settings. We mapped each measure to the interval by scaling each measure with meaningful upper and lower bounds:

 Mscaled(M)=M−MinfMsup−Minf

For MAE, we set to the lowest observed MAE of all models and . For NF, we set and . For MEC, we set and to the highest observed MEC of all models. For IAS, we set and to the highest observed IAS of all models. To combine the three measures in a single dimension, we mapped interpretability ad-hoc as: . This weights all three (scaled) measures equally. The maximum interpretability is 3 for the constant model that always predicts the median wine quality. The theoretical minimum interpretability is 0 for a model that uses all features and has the highest interaction strength and highest main effect complexity measure among all Pareto optimal models. Figure 3 maps each model and hyperparameter configuration from the Pareto set to the performance / interpretability space.

## 6 Discussion

We proposed three model-agnostic measures for machine learning model complexity based on functional decomposition: number of features used, interaction strength and main effect complexity. Due to their model-agnostic nature, the measures allow model comparison across different model classes. We argued that minimizing these measures for a machine learning model improves its post-hoc interpretation. We demonstrated that the measures can be optimized directly with multi-objective optimization to make the trade-off between interpretability and performance explicit. Our proposed measures can be applied to a wide range of models since they are model-agnostic and work for regression and binary classification (based on classification scores / probabilities). We formulated the measures with both continuous and categorical features in mind, but leave an in-depth investigation of categorical features open for future work. The measures can be used for model selection, for model benchmarks and as objectives in automated machine learning frameworks.

#### 6.0.1 Limitations.

The proposed decomposition of the prediction function and definition of the complexity measures will not be appropriate in every situation. For example, all higher order effects are combined into a single interaction strength measure that does not distinguish between two-way interactions and higher order interactions. Two models can have the same IAS, but one has a single two-way interaction, the other many different higher order interactions. However, the framework of ALE decomposition allows to estimate higher order effects and to construct different interaction measures. The main effect complexity measure only considers linear segments but not e.g. seasonal components or other structures. Furthermore, the complexity measures quantify machine learning models from a functional point of view and ignore the structure of the model (e.g. whether it can be represented by a tree).

#### 6.0.2 The bigger picture.

Interpretability is a high-dimensional concept (sparsity, additivity, fidelity, human simulability, …) and we need several approaches to make interpretability measurable. In this context we see our work complementary to other approaches [15, 10, 27], which together form a basis for a more rigorous definition of interpretability as demanded by [11, 23]. Availability of different interpretability measures also fits in with the notion that interpretability depends on the audience and the context [30]. Different situations require differently weighted interpretability measures. In some situations we might prefer sparseness and a lack of interactions, in others it might be important that we can represent the model as a decision list. A multi-dimensional view of interpretability solves the lack of definition of interpretability and supports researcher to make quantified, verifiable and clearer statements about interpretability.

#### 6.0.3 Implementation.

The code for this paper is available at https://github.com/compstat-lmu/paper_2019_iml_measures. For the examples and experiments we relied on the mlr package [6] in R [28].

#### 6.0.4 Acknowledgements.

This work is funded by the Bavarian State Ministry of Science and the Arts in the framework of the Centre Digitisation.Bavaria (ZD.B) and supported by the German Federal Ministry of Education and Research (BMBF) under Grant No. 01IS18036A. The authors of this work take full responsibilities for its content.