Tropical Data Science

Outline

Introduction
Data Science Overview
1. Supervised Learning (Inferential Statistics)
  1. Classifications
  2. Regression
2. Unsupervised Learning (Descriptive Statistics)
Phylogenetics to Phylogenomics
1. Phylogenetic Trees
2. Space of Phylogenetic Trees
Basics in Tropical Geometry
Tropical Unsupervised Learning (Tropical Descriptive Statistics)
1. Tropical Fermat Weber Points
2. Tropical Frécet Means
3. Tropical Principal Component Analysis (PCA)
Tropical Supervised Learning (Tropical Inferential Statistics)
1. Tropical Classifications
  1. Tropical Support Vector Machines
  2. Tropical Linear Discriminant Analysis
2. Tropical Regression

Key Words: Machine Learning Models, Max-Plus Algebra, Phylogenetics, Phylogenomics, Tropical Geometry, Ultrametrics.

1 Introduction

Due to increasing amount of data today, data science is one of most exciting fields. It finds applications in statistics, computer science, business, biology, data security, physics, and so on. Most of statistical models in data sciences assume that data points in an input sample are distributed over a Euclidean space if they have numerical measurements. However, in some cases this assumption can be failed. For example, a space of phylogenetic trees with a fixed set of leaves is an union of lower dimensional cones over $R^{e}$ , where $e = (\frac{N}{2})$ with $N$ is the number of leaves [2]. Since the space of phylogenetic trees is an union of lower dimensional cones, we cannot just apply statistical models in data science to a set of phylogenetic trees [20].

There has been much work in spaces of phylogenetic trees. In 2001, Billera-Holmes-Vogtman (BHV) developed a notion of a space of phylogenetic trees with a fixed set of labels for leaves [4], which is a set of all possible phylogenetic trees with the fixed set of lebels on leaves and is an union of orthants, each orthant is for all possible phylogenetic trees with a fixed tree topology. In this space, two orthants are next to each other if the tree topology for one orthant is one nearest neighbor interchange (NNI) distance to the tree topology for the other orthant. They also showed that this space is CAT(0) space so that there is a unique shortest connecting paths, or geodesics, between any two points in the space defined by the $C A T (0)$ -metric. There is some work in development on machine learning models with the BHV metric. For example, Nye defined a notion of the first order principal component geodesic as the unique geodesic with the BHV metric over the the tree space which minimizes the sum of residuals between the geodesic and each data point [14]. However, we cannot use a convex hull under the BHV metric for higher principal components because Lin et. al showed that the convex hull of three points with the BHV metric over the tree space can have arbitrarily high dimension [10].

In 2004, Speyer and Sturmfels showed a space of phylogenetic trees with a given set of labels on their leaves is a tropical Grassmanian [18], which is a tropicalization of a linear space defined by a set of linear equations [20] with the max-plus algebra. The tropical metric with max-plus algebra on the tree space is known to behave very well [1, 6]. For example, contrarily to the BHV metric, the dimension of the convex hull of $s$ tropical points is at most $s - 1$ .

Thus, this paper focuses on the tropical metric over tree spaces. In this paper we review some development on statistical learning models with the tropical metric with max-plus algebra on tree spaces as well as the tropical projective space, and we overview some open problems.

2 Data Science Overview

In this section, we briefly overview statistical models in data science. For more details, we recommend to read Introduction of Statistical Learning with R http://faculty.marshall.usc.edu/gareth-james/ISL/.

In data science there are roughly two sub-branches of data science: unsupervised learning and supervised learning (Figure 1

). In unsupervised learning, our goal is to compute a descriptive statistics to see how data points are distributed over the sample space or how data points are clustered together. In statistics, unsupervised learning corresponds to descriptive statistics. In supervised learning, our goal is to predict/infer the response variable from explanatory variables. In statistics, supervised learning corresponds to inferential statistics. Like unsupervised learning and supervised learning, there are some notations with different names between machine learning and statistics. Thus we summarize some of the differences in Table

Statistics	Data Science
descriptive statistics	Unsupervised learning
inferential statistics	Supervised learning
response variable	target variable
explanatory variable	predictor variable
	feature

Table 1: There are several notations with different names in statistics and data science.

2.1 Basic Definitions

Response variable – the variable for an interest in a study or experiment. It can be called as a dependent variable. In machine learning it is also called a target variable.
Explanatory variable – the variable explains the changes in the response variable. It can be also called a feature or independent variable. In machine learning it is also called feature or predictor.

2.2 Unsupervised Learning

Since unsupervised learning is descriptive, there is no response variables. In unsupervised learning, we try to learn how data points are distributed and how they related to each other. Among them, there are mainly two categories: clustering and dimensionality reduction.

Clustering – grouping data points into subsets by their “similarity”. These similarities are defined by a user. These groups are called clusters.
Dimensionality reduction – reducing the dimension of data points with minimizing the loss of information. One of the most commonly used methods is principal component analysis (PCA), a dimension reduction procedure via linear algebra.

2.3 Supervised Learning

Supervised learning is inferential. Thus, there are the response variable and explanatory variables in an input data set. Depending on the scale of the response variable, we can separate two groups in supervised learning: classification and regression. In classification, the response variable has categorical scale and in regression, the response variable has numerical (interval) scale.

Classifications
– the response variable is categorical. Under classification, there are algorithms like logistic regression, support vector machine, linear discriminant analysis, classification trees, random forests, adaboost and etc.
Regression
– the response variable is numerical. There are algorithms like linear regression, regression trees, lasso, ridge regression, random forests, adaboost and etc.

For more details, see the following papers:

Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani. Introduction of Statistical Learning with R http://faculty.marshall.usc.edu/gareth-james/ISL/.

3 Phylogenetics to Phylogenomics

In this section we overview basics in phylogenetics and basic problem for phylogenomics.

3.1 Phylogenetic Trees

Evolutionary, or phylogenetic trees, show an organism’s evolutionary relationships over time, through the use of tree diagrams. Phylogenetic trees still consist of vertices (nodes) and edges (branches). Each node in a phylogenetic tree represents a past or present taxon or population: exterior nodes in a phylogenetic tree represent taxon or population at present; and interior nodes represent their ancestors. Edges in a phylogenetic tree have weights and a weight in each edge represents mutation rates multiplied by evolutional time from its ancestor to a taxon.

In Figure 2, an exterior vertex (leaf or tip) represents the current taxa ( $V = {S p e c i e s 1, S p e c i e s 2, S p e c i e s 3})$ . An interior vertex represents an extinct taxa $(V = {4, 5})$ where ancestors split into two subgroups. Vertices and edges can still be labeled; however, only the leaves or tips are labeled in a phylogenetic tree. This is due to the past taxa often being inferred and not exactly known. Vertices in phylogenetic trees can be DNA sequences, shared genes or interrelated species, depending on the context of the tree. The root $(V = {5})$ of the tree now represents the common ancestor of all leaves, $S p e c i e s 1$ , $S p e c i e s 2$ , and $S p e c i e s 3$ .

Figure 2: Example Rooted Binary Phylogenetic Tree

Phylogenetic trees are trees. Thurs, they remain acylic and connected. In terms of evolutional biology, these properties are intuitive, as a species must evolve from something and also as time progresses species can only evolve forward. Weights on edges in a tree represent the notion of time. The distance $(d_{S p e c i e s 1, S p e c i e s 2})$ measures the dissimilarity between Species 1 and Species 2 with respect to time.

Let $N$ be the number of leaves on a phylogenetic tree. If a total of weights of all edges in a path from the root to each leaf $i \in [N] := {1, \dots, N}$ in a rooted phylogenetic tree $T$ is the same for all leaves $i \in [N]$ , then we call a phylogenetic tree $T$ equidistant tree. The height of an equidistant tree is the total weight of all edges in a path from the root to each leaf in the tree.

3.1.1 Phylogenetic Tree Reconstruction

Phylogenetic reconstruction uses genetic data to create an inferential evolutionary (phylogenetic) tree. These changing characters are the mutations in DNA sequences. DNA sequences represent a shared gene across multiple species. Trees are excellent at representing the evolutionary changes of this shared gene through node splits and leaves.

Even though we do not discuss details on a phylogenetic tree reconstruction in this paper, multiple steps and techniques are involved in the reconstruction process and there are several types of tree reconstruction methods;

Maximum likelihood estimation (MLE) methods
– These methods describe evolution in terms of a discrete-state continuous-time Markov process.
Maximum Parsimony – Reconstructs tree with the least evolutionary changes which explain data.
Bayesian inference for trees
– Use Bayes Theorem and MCMC to estimate the posterior distribution rather than obtaining the point estimation.
Distance based methods – Reconstructing a tree from a distance matrix.

3.2 Space of Phylogenetic Trees

There are several ways to define a space of phylogenetic trees with different metrics. One of the very well-known tree spaces is Billera-Holmes-Vogtmann tree space. In 2001, Billera-Holmes-Vogtmann (BHV) introduced a continuous space which models the set of rooted phylogenetic trees with edge lengths on a fixed set of leaves. In this space, edge lengths in a tree are continuous and we assign a coordinate for each interior edge. Note that unrooted trees can be accommodated by designating a fixed leaf node as the root. The BHV tree space is not Euclidean, but it is non-positively curved, and thus has the property that any two points are connected by a unique shortest path through the space, called a geodesic. The distance between two trees is defined as the length of the geodesic connecting them. While in this paper, we do not consider the BHV tree space, read [4] for interested readers.

Through this paper, we assume that all phylogenetic trees are equidistant trees. An equidistance tree is a rooted phylogenetic tree such that the sum of all branch lengths in the unique path from the root to each leaf in the tree, called the height of the tree, is fixed and they are the same for all leaves in the tree. In phylogenetics this assumption is fairly mild since the multispecies coalescent model assumes that all gene trees have the same height.

Example 1.

Suppose $N = 4$ . Consider two rooted phylogenetic trees with the set of labels on the leaves ${a, b, c, d}$ in Figure 3. Note that for each tree, the sum of branch lengths in the unique path from the root to each leaf is $1$ . Therefore they are equidistant trees with their height are equal to $1$ .

Figure 3: Examples of equidistant trees with $N = 4$ leaves with the set of labels ${1, b, c, d}$ and with their height equal to $1$ .

For the space of equidistant trees with the fixed set of labels on their leaves, the BHV tree space might not be appropriate [7]. Therefore, we consider the space of ultrametrics. To define ultrametrics and theire relations to equidistant trees, we need to define dissimilarity maps.

Definition 2.

[Dissimilarity Map] A dissimilarity map $w$ is a function $w : [N] \times [N] \to R_{\geq 0}$ such that

w (i, j) = {\begin{matrix} 0 & if i = j \geq 0 & else, \end{matrix}

for all $i, j \in [N]$ . If a dissimilarity map $w$ additionally satisfies the triangle inequality, that is:

w (i, j) \leq w (i, k) + w (k, j),

for all $i, j, k \in [N]$ , then $w$ is called a metric. If there exists a phylogenetic tree $T$ such that $w (i, j)$ coincides with the total branch length of the edges in the unique path from a leaf $i$ to a leaf $j$ for all leaves $i, j \in [N]$ , then we say $w$ a tree metric. If a metric $w$ is a tree metric and $w (i, j)$ is the total branch length of all edges in the path from a leaf $i$ to a leaf $j$ for all leaves $i, j \in [N]$ in a phylogenetic tree $T$ , then we say $w$ realises a phylogenetic tree $T$ or $w$ is a realisable of a phylogenetic tree $T$ .

Since

w (i, j) = {\begin{matrix} 0 & if i = j w (j, i) & else, \end{matrix}

to simplify we write

w = (w (1, 2), w (1, 3), \dots, w (N - 1, N)) .

Example 3.

We consider equidistant trees in Figure 3. The dissimilarity map obtained from the left tree in Figure 3 is

(1.2, 1.8, 2, 1.8, 2, 2) .

Similarly, the dissimilarity map obtained from the right tree in Figure 3 is

(0.2, 2, 2, 2, 2, 1) .

Since these dissimilarity maps are obtained from phylogenetic trees, they are tree metrics.

Definition 4 (Three Point Condition).

If a metric $w$ satisfies the following condition: For every distinct leaves $i, j, k \in [N]$ ,

max {w (i, j), w (i, k), w (j, k)}

achieves twice, then we say that $w$ satisfies the three point condition.

Definition 5 (Ultrametrics).

If a metric $w$ satisfies the three point condition then $w$ is called an ultrametric.

Theorem 6 ([8]).

A dissimilarity map $w : [N] \times [N]$ is ultrametric if and only if $w$ is realisable of an equidistant tree with labels $[N]$ . In addition, for each equidistant tree there exists a unique ultrametric. Conversely, for each ultrametric, there exists a unique equidistant tree.

Example 7.

We again consider equidistant trees in Figure 3. The dissimilarity map obtained from the left tree in Figure 3 is

(1.2, 1.8, 2, 1.8, 2, 2) .

Similarly, the dissimilarity map obtained from the right tree in Figure 3 is

(0.2, 2, 2, 2, 2, 1) .

Since these phylogenetic trees are equidistant trees, these dissimilarity maps are ultrametrics by Theorem 6.

From Theorem 6 we consider the space of ultrametrics with labels $[N]$ as a space of all equidistant trees with the label set $[N]$ . Let $U_{N}$ be the space of ultrametrics for equidistant trees with the leaf labels $[N]$ . In fact we can write $U_{N}$ as the tropicalization of the linear space generated by linear equations.

Let $L_{N} \subseteq R^{e}$ be the linear subspace defined by the linear equations such that

x_{i j} - x_{i k} + x_{j k} = 0

(1)

for $1 \leq i < j < k \leq N$ . For the linear equations (1) spanning the linear space $L_{N}$ , the max-plus tropicalization $T r o p (L_{N})$ of the linear space $L_{N}$ is the tropical linear space with $w \in R^{e}$ such that

max {w_{i j}, w_{i k}, w_{j k}}

achieves at least twice for all $i, j, k \in [N]$ . Note that this is exactly the three point condition defined in Definition 5.

Theorem 8.

[20, Theorem 2.18] The image of $U_{N}$ in the tropical projective torus $R^{n} / R 1$ coincides with $trop (L_{N})$ .

For example, if $N = 4$ , The space of ultrametrics $U_{4}$ is a two-dimensional fan with $15$ maximal cones.

For more details, see the following papers:

C. Semple and M. Steel. Phylogenetics, [17].
Lin et al. Convexity in Tree Spaces [11].

4 Basics in Tropical Geometry

Here we review some basics of tropical arithmetic and geometry, as well as setting up the notation through this paper.

Definition 9 (Tropical arithmetic operations).

Throughout this paper we perform arithmetic over the max-plus tropical semiring $(R \cup {- \infty}, ⊞, ⊙)$ . Over this tropical semiring, the basic tropical arithmetic operations of addition and multiplication are defined as the following:

a ⊞ b := max {a, b}, a ⊙ b := a + b where a, b \in R \cup {- \infty} .

Over this tropical semiring, $- \infty$ is the identity element under addition and $0$ is the identity element under multiplication.

Example 10.

Suppose we have $a = 1, b = - 3$ . Then

\begin{matrix} 1 ⊞ (- 3) & = & max {1, - 3} = 1 1 ⊙ (- 3) & = & 1 + - 3 = - 2. \end{matrix}

Definition 11 (Tropical scalar multiplication and vector addition).

For any $a, b \in R \cup {- \infty}$ and for any $v = (v_{1}, \dots, v_{e}), w = (w_{1}, \dots, w_{e}) \in (R \cup - {\infty})^{e}$ , tropical scalar multiplication and tropical vector addition are defined as:

a ⊙ v = (a + v_{1}, a + v_{2}, \dots, a + v_{e})

a ⊙ v ⊞ b ⊙ w = (max {a + v_{1}, b + w_{1}}, \dots, max {a + v_{e}, b + w_{e}}) .

Example 12.

Suppose we have

\begin{matrix} v & = & (1, 2, 3), w & = & (3, - 2, 1), \end{matrix}

and $a = 1, b = - 3$ . Then we have

\begin{matrix} a ⊙ v & = & (1 + 1, 1 + 2, 1 + 3) = & (2, 3, 4), \end{matrix}

and

\begin{matrix} a ⊙ v ⊞ b ⊙ w & = & (max {1 + 1, (- 3) + 3}, max {1 + 2, (- 3) + (- 2)}, max {1 + 3, (- 3) + 1}) = & (2, 3, 4) . \end{matrix}

Throughout this paper we consider the tropical projective torus, that is, the projective space $R^{e} / R 1$ , where $1 := (1, 1, \dots, 1)$ , the all-one vector.

Example 13.

Consider $R^{e} / R 1$ . Then let

v = (1, 2, 3) .

Then over $R^{e} / R 1$ we have the following equality:

v = (1, 2, 3) = (0, 1, 2) .

Note that $R^{e} / R 1$ is isometric to $R^{e - 1}$ .

Example 14.

Consider $R^{e} / R 1$ . Then let

v = (1, 2, 3), w = (1, 1, 1) .

Also let $a = - 1, b = 3$ . Then we have

a ⊙ v ⊞ b ⊙ w = (max (- 1 + 1, 3 + 1), max (- 1 + 2, 3 + 1), max (- 1 + 3, 3 + 1)) = (4, 4, 4) = (0, 0, 0) .

In order to conduct a statistical analysis we need a distance measure between two vectors in the space. Thus we discuss a distance between two vectors in the tropical projective space. In fact the following distance between two vectors in the tropical projective space is a metric.

Definition 15 (Generalized Hilbert projective metric).

For any two points $v, w \in R^{e} / R 1$ , the tropical distance $d_{t r} (v, w)$ between $v$ and $w$ is defined:

d_{t r} (v, w) = max i, j {| v_{i} - w_{i} - v_{j} + w_{j} | : 1 \leq i < j \leq e} = max i {v_{i} - w_{i}} - min i {v_{i} - w_{i}},

(2)

where $v = (v_{1}, \dots, v_{e})$ and $w = (w_{1}, \dots, w_{e})$ . This distance is a metric in $R^{e} / R 1$ . Therefore, we call $d_{t r}$ tropical metric.

Example 16.

Suppose $u_{1}, u_{2} \in R^{3} / R 1$ such that

u_{1} = (0, 0, 0), u_{2} = (0, 3, 1) .

Then the tropical distance between $u_{1}, u_{2}$ is

d_{t r} (u_{1}, u_{2}) = max (0, - 3, - 1) - min (0, - 3, - 1) = 0 - (- 3) = 3.

Similar to the BHV metric over the BHV tree space, we need to define a geodesic over the space of ultrametrics. In order to define a tropical geodesic we need to define a tropical polytope:

Definition 17.

Suppose we have a finite subset $V = {v_{1}, \dots, v_{s}} \subset R^{e}$ The tropical convex hull or tropical polytope of $V$ is the smallest tropically-convex subset containing $V \subset R^{e}$ written as the set of all tropical linear combinations of $V$ such that:

t c o n v (V) = {a_{1} ⊙ v_{1} ⊞ a_{2} ⊙ v_{2} ⊞ \dots ⊞ a_{s} ⊙ v_{s},

where $v_{1}, \dots, v_{s} \in V and a_{1}, \dots, a_{s} \in R}$ . A tropical line segment between two points $v_{1}, v_{2}$ is a tropical convex hull of two points ${v_{1}, v_{2}}$ .

Note that the length between two points $u_{1}, u_{2} \in R^{3} / R 1$ along the tropical line segment between $u_{1}, u_{2}$ equals to the tropical distance $d_{t r} (u_{1}, u_{2})$ . In this paper we define a tropical line segment between two points as a tropical geodesic between these points.

Example 18.

Suppose $u_{1}, u_{2} \in R^{3} / R 1$ such that

u_{1} = (0, 0, 0), u_{2} = (0, 3, 1) .

From the previous example, the tropical distance between $u_{1}, u_{2}$ is

d_{t r} (u_{1}, u_{2}) = 3.

Also the tropical line segment between $u_{1}, u_{2}$ is a line segment between these three points:

\begin{matrix} (0, 0, 0) (0, 2, 0) (0, 3, 1) . \end{matrix}

The length of the line segment is

d_{t r} ((0, 0, 0), (0, 2, 0)) + d_{t r} ((0, 2, 0), (0, 3, 1)) = 2 + 1 = 3.

Example 19.

Suppose we have a set $V = {v_{1}, v_{2}, v_{3}} \subset R^{3} / R 1$ where

v_{1} = (0, 0, 0), v_{2} = (0, 3, 1), v_{3} = (0, 2, 5) .

Then we have the tropical convex hull $t c o n v (V)$ of $V$ is shown in Figure 4.

Figure 4: Tropical polytope of three points $(0, 0, 0), (0, 3, 1), (0, 2, 5)$ in $R^{3} / R 1$ .

For more details, see the following papers:

D. Maclagan and B. Sturmfels. Introduction to Tropical Geometry [9].

5 Tropical Unsupervised Learning

Unsupervised learning is descriptive and we do not know much about descriptive statistics using tropical geometry with max-plus algebra, for example, tropical Fermat Weber (FW) points and tropical Frécet means.

In this section we discuss tropical FW points and tropical Frécet means, what they are and what we know and we do not know. In the end of this section, we discuss tropical principal component analysis (PCA). Over this section we consider the tropical projective torus $R^{e} / R 1$ .

5.1 Tropical Fermat Weber Points

Suppose we have a sample ${v_{1}, \dots, v_{s}}$ over $R^{e} / R 1$ . A tropical Fermat-Weber point $y$ minimizes the sum of distances to the given points.

y := a r g m i n z \in R^{e} / R 1 s \sum i = 1 d_{t r} (z, v_{i}) .

(3)

There are properties of tropical Fermat-Weber points of a sample ${v_{1}, \dots, v_{s}}$ over $R^{e} / R 1$ .

Proposition 20.

Suppose $M = R^{e} / R 1$ . Then the set of tropical Fermat-Weber points of a sample ${v_{1}, \dots, v_{s}}$ over $R^{e} / R 1$ is a convex polytope. It consists of all optimal solutions $y = (y_{1}, \dots, y_{e})$

to the following linear program:

minimized1+d2+⋯+ds\rm subject% toyj−yk−vij+vik≥−difor alli=1,…,sand1≤j,k≤e,yj−yk−vij+vik≤−difor% alli=1,…,sand1≤j,k≤e.

(4)

From Proposition 20, there can be infinitely many tropical Fermat-Weber points of a sample.

If we focus on the space of ultrametrics $U_{N}$ for equidistance trees with $N$ leaves, then we have the following proposition:

Proposition 21.

If a sample ${v_{1}, \dots, v_{s}}$ over the space of ultrametrics $U_{N}$ , then tropical Fermat-Weber points are in $U_{N}$ .

In [12], we showed explicitly how to compute the set of all possible Fermat-Weber points in $R^{e} / R 1$ . However, we do not know the minimal set of inequalities needed to define the set of all tropical Fermat-Weber points of a given sample. Thus here is an open problem:

Problem 22.

What is the minimal set of inequalities needed to define the set of all tropical Fermat-Weber points of a given sample? What is the time complexity to compute the set of tropical Fermat-Weber points of a sample of $m$ points in $R^{e} / R 1$ ? Is there a polynomial time algorithm to compute the vertices of the polytope of tropical Fermat-Weber points of a sample of $s$ points in $R^{e} / R 1$ in $s$ and $e$ ?

For more details, see the following papers:

B. Lin and R. Yoshida Tropical Fermat–Weber Points [12].

5.2 Tropical Frécet Means

Suppose we have a sample ${v_{1}, \dots, v_{s}}$ over $R^{e} / R 1$ . A tropical Fréchet mean $y$ minimizes the sum of distances to the given points.

y := a r g m i n z \in R^{e} / R 1 s \sum i = 1 d_{t r} (z, v_{i})^{2} .

(5)

As we formulated computing a tropical Fermat-Weber point as a linear programming problem, we can also formulate computing a tropical Frécet mean as a quadratic programming problem:

minimized21+d22+⋯+d2s\rm subject toyj−yk−vij+vik≥−difor alli=1,…,sand1≤j,k≤e,yj−yk−vij+vik≤−difor% alli=1,…,sand1≤j,k≤e.

(6)

While we know some propertied of tropical Fermat-Weber points we do not know much about tropical Fréchen means. Here are some basics on tropical Fréchet means.

Proposition 23.

Suppose $M = R^{e} / R 1$ . Then the set of tropical Fréchen means of a sample ${v_{1}, \dots, v_{s}}$ over $R^{e} / R 1$ is a convex polytope. It consists of all optimal solutions $y = (y_{1}, \dots, y_{e})$ to the following quadratic program:

minimized21+d22+⋯+d2s\rm subject toyj−yk−vij+vik≥−difor alli=1,…,sand1≤j,k≤e,yj−yk−vij+vik≤−difor% alli=1,…,sand1≤j,k≤e.

(7)

Still we do not know much about tropical Fréchet means. First we have the following problem.

Problem 24.

If a sample ${v_{1}, \dots, v_{s}}$ over the space of ultrametrics $U_{N}$ , then are tropical Féchet means in $U_{N}$ ?

We still do not know how to compute tropical Fréchet means in efficient ways. So we have the following problem:

Problem 25.

Suppose we have ${v_{1}, \dots, v_{s}}$ over $R^{e} / R 1$ . Is there an algorithm to compute all tropical Fréchet means in $R^{e} / R 1$ ?

5.3 Tropical Principal Component Analysis (PCA)

Principal component analysis (PCA) is one of the most popular methods to reduce dimensionality of input data and to visualize them. Classical PCA takes data points in a high-dimensional Euclidean space and represents them in a lower-dimensional plane in such a way that the residual sum of squares is minimized. We cannot directly apply the classical PCA to a set of phylogenetic trees because the space of phylogenetic trees with a fixed number of leaves is not Euclidean; it is a union of lower dimensional polyhedral cones in $R^{(\frac{N}{2})}$ , where $N$ is the number of leaves.

There is a statistical method similar to PCA over the space of phylogenetic trees with a fixed set of leaves in terms of the Billera-Holmes-Vogtman (BHV) metric.

In 2001, Billera-Holmes-Vogtman developed the space of phylogenetic trees with fixed labeled leaves and they showed that it is $C A T (0)$ space [5]. Therefore, a geodesic between any two points in the space of phylogenetic trees is unique.

Short after that, Nye showed an algorithm in [15] to compute the first order principal component over the space of phylogenetic trees of $N$ leaves with the BHV metric.

Nye in [15] used a convex hull of two points, i.e., the geodesic, on the tree space as the first order PCA. However, this idea can not be generalized to higher order principal components with the BHV metric since the convex hull of three points with the BHV metric over the tree space can have arbitrarily high dimension [11].

On the other hand, the tropical metric in the tree space in terms of the max-plus algebra is well-studied and well-behaved [13]. For example, the dimension of the convex hull of $s$ points in terms of the tropical metric is at most $s - 1$ . Using the tropical metric, Yoshida et al. in [20] introduced a statistical method similar to PCA with the max-plus tropical arithmetic in two ways: the tropical principal linear space, that is, the best-fit Stiefel tropical linear space of fixed dimension closest to the data points in the tropical projective torus; and the tropical principal polytope, that is, the best-fit tropical polytope with a fixed number of vertices closest to the data points. The authors showed that the latter object can be written as a mixed-integer programming problem to compute them, and they applied the second definition to datasets consisting of collections of phylogenetic trees. Nevertheless, exactly computing the best-fit tropical polytope can be expensive due to the high-dimensionality of the mixed-integer programming problem.

Definition 26.

Let $P=tconv\,(D(1),…,D(s))⊆Re/R1$ be a tropical polytope with its vertices ${D^{(1)}, \dots, D^{(s)}} \subset R^{e} / R 1$ and let $S = {u_{1}, \dots u_{n}}$ be a sample from the space of ultrametrics $U_{N}$ . Let $Π_{P} (S) := \sum_{i = 1}^{| S |} d_{t r} (u_{i}, u_{i}^{'})$ , where $u_{i}^{'}$ is the tropical projection of $u_{i}$ onto a tropical polytope $P$ . Then the vertices $D^{(1)}, \dots, D^{(s)}$ of the tropical polytope $P$ are called the $(s - 1)$ -th order tropical principal polytope of $S$ if the tropical polytope $P$ minimizes $Π_{P} (S)$ over all possible tropical polytopes with $s$ many vertices.

In [16]

, Page et.al developed a heuristic method to compute tropical principal polytope and they applied it to empirical data sets on genome data of influenza flu collected from New York city, Apicomplexa, and African coelacanth genome data sets.

Also Page et.al showed the following theorem and lemma:

Theorem 27 ([16]).

Let $P=tconv\,(D(1),…,D(s))⊆Re/R1$ be a tropical polytope spanned by ultrametrics in $U_{N}$ . Then $P \subseteq U_{N}$ and any two points $x$ and $y$ in the same cell of $P$ are also ultrametrics with the same tree topology.

Lemma 28 ([16]).

Let $P=tconv\,(D(1),…,D(s))⊆Re/R1$ be a tropical polytope spanned by ultrametrics. The origin $0$ is contained in $P$ if and only if the path between each pair of leaves $i, j$ passes through the root of some $D^{(i)}$ .

There are still some open problem on tropical PCA. Here is one of questions we can work on:

Conjecture 29.

There exists a tropical Fermat-Weber point $x^{*} \in U_{N}$ of a sample $D^{(1)}, \dots, D^{(n)}$ of ultrametric trees which is contained in the $s$ th order tropical PCA of the dataset for $s \geq 1$ .

For more details, see the following papers:

R. Yoshida, L. Zhang, and X. Zhang. Tropical Principal Component Analysis and its Application to Phylogenetics. [20].
R. Page, R. Yoshida, and L. Zhang. Tropical principal component analysis on the space of ultrametrics. [16].

6 Tropical Supervised Learning

For tropical supervised learning, there is not much done. For classification, there is some work done. Recently Tang et.al in [19] introduced a notion of tropical support vector machines (SVMs). In this section we discuss tropical SVMs and we introduce a notion of tropical linear discriminant analysis (LDA).

6.1 Tropical Classifications

For tropical classification, we consider the binary response variables. Suppose we have a data set given that

{(x_{1}, y_{1}), \dots, (x_{n}, y_{n})},

where $x_{1}, \dots, x_{n} \in R^{e} / R 1$ and $y_{1}, \dots, y_{n} \in {0, 1}$ . Therefore, the response variable $y_{i}$ is binary. Thus, we can partition a sample of data points $x_{1}, \dots, x_{n} \in R^{e} / R 1$ into two sets $P$ and $Q$ such that

{\begin{matrix} x_{i} \in P & if y_{i} = 0, x_{i} \in Q & if y_{i} = 1. \end{matrix}

6.1.1 Tropical support vector machine SVMs

A support vector machine (SVM) is a supervised learning model to predict the categorical response variable. For a binary response variable, a classical linear SVM classifies data points by finding a linear hyperplane to separate the data points into two groups. In this paper we refer a classical SVM as a classical linear SVM over an Euclidean space

R^{e}

with

L_{2}

norm.

For an Euclidean space $R^{e}$ , there are two types of SVMs: hard margin SVMs and soft margin SVMs. A hard margin SVM is a model with the assumption that all data points can be separated by a linear hyperplane into two groups without errors. A soft margin SVM is a model which maximizes the margin and also allows some data points in the wrong side of the hyperplane.

Similar to a classical SVM over a Euclidean space, a tropical SVM is a supervised learning model which classifies data points by finding a tropical hyperplane to separate them. In [19], as a classical SVM, Tang et.al defined two types of tropical SVMs: hard margin tropical SVMs and soft margin tropical SVMs. A hard margin tropical SVM introduced by [3] is, similar to a classical hard margin SVM, a model to find a tropical hyperplane which maximizes the margin, the minimum tropical distance from data points to the tropical hyperplane (which is $z$ in Figure 5), to separate these data points into open sectors. Note that an open sector of a tropical hyperplane can be seen as a tropical version of an open half space defined by a hyperplane. A tropical soft margin SVM introduced by [19] is a model to find a tropical SVM to maximizes the margin but it also allows some data points into a wrong open sector.

The authors in [3] showed that computing a tropical hyperplane for a tropical hard margin SVM from a given sample on the tropical projective space can be formulated as a linear programming problem. Again, note that, similar to the classical hard margin SVMs, hard margin tropical SVMs assume that there exists a tropical hyperplane such that it separates all data points in the tropical projective space into each open sector (see the left figure in Figure 5).

Figure 5: A hard margin tropical SVM (LEFT) and a soft margin tropical SVM (RIGHT) with the binary response variable. A hard margin tropical SVM assumes that all data points from the given sample can be separated by a tropical hyperplane. Red squared dots are data points from $P$ and blue circle dots are data points from $Q$ . A tropical hard margin hyperplane for a tropical hard margin tropical SVM is obtained by maximizing the margin $z$ in the left figure, the distance from the closest data point from the tropical hyperplane (the width of the grey area from the tropical hyperplane in the left figure). A soft margin tropical hyplerplane for a soft margin tropical SVM is obtained by maximizing a margin similar to a hard margin tropical SVM and by minimizing the sum of $α$ and $β$ at the same time.

In order to discuss details on tropical SVMs, we need to define a tropical hyplerplane and their open sectors.

Definition 30.

Suppose $ω := (ω_{1}, \dots, ω_{e}) \in R^{e} / R 1$ . The tropical hyperplane defined by $ω$ , denoted by $H_{ω}$ , is the set of all points $x \in R^{e} / R 1$ such that

max {ω_{1} + x_{1}, \dots, ω_{e} + x_{e}}

is attained at least twice. $ω$ is called the normal vector of $H_{ω}$ .

Definition 31.

A tropical hyperplane $H_{ω}$ divides the tropical projective space $R^{e} / R 1$ into $e$ components. These $e$ components divided by $H_{ω}$ are called open sectors given that:

S_{ω}^{i} := {x \in R^{e} / R 1 | ω_{i} + x_{i} > ω_{j} + x_{j}, \forall j \neq i}, i = 1, \dots, e .

Example 32.

Consider $R^{3} / R 1$ . Then a tropical hypoerplane in $R^{3} / R 1$ has three open sectors seen as Figure 5. Note that $R^{3} / R 1$ is isometric to $R^{2}$ .

Now we define the tropical distance from a point to a tropical hyperplane.

Definition 33.

The tropical distance from a point $x \in R^{e} / R 1$ to the tropical hyperplane $H_{ω}$ is defined as:

d_{t r} (x, H_{ω}) := min {d_{t r} (x, y) | y \in H_{ω}} .

A tropical hard margin SVM assumes that all points are separated by a tropical hyperplane and all data points with the same category for their response variable are assigned in the same open sector. Thus, to compute a tropical hard margin hyperplane for a tropical SVM, we Want to find the normal vector $ω$ of a tropical hyperplane $H_{ω}$ such that

\begin{matrix} max ω \in R^{d} min ξ \in P \cup Q {ξ_{i (ξ)} + ω_{i (ξ)} - ξ_{j (ξ)} - ω_{j (ξ)}} s.t. & \forall ξ \in P \cup Q, \forall l \neq i (ξ), j (ξ), (ξ + ω)_{l} \leq (ξ + ω)_{j (ξ)} \leq (ξ + ω)_{i (ξ)}, \end{matrix}

where $i (ξ)$ and $j (ξ)$ are the largest and the second largest coordinate of the vector $ξ + ω$ for all $ξ \in P \cup Q$ .

Theorem 34 ([19]).

The normal vector $ω$ of the tropical hard margin $H_{ω}$ for a tropical SVM is the optimal solution of the following linear programming problem:

	$max z \in R z$	(8)
$s.t. \forall ξ \in P \cup Q,$	$z+\textcolorblackξj(ξ)+ωj(ξ)\textcolorblack−ξi(ξ)−ωi(ξ)≤0,$	(9)
$\forall ξ \in P \cup Q,$	$ω_{j (ξ)} - ω_{i (ξ)} \leq ξ_{i (ξ)} - ξ_{j (ξ)},$	(10)
$\forall ξ \in P \cup Q,$	$\forall l \neq i (ξ), j (ξ), ω_{l} - ω_{j (ξ)} \leq ξ_{j (ξ)} - ξ_{l} .$	(11)

As we discussed earlier, tropical soft margin SVMs are similar to tropical hard margin SVMs. They try to find a tropical hyperplane which maximizes the margin but also they allow some points to be in a wrong open sector by introducing extra variables $α, β$ in Figure 5. Tang et.al showed in [19] that a soft margin tropical hyperplane for a tropical SVM is the optimal solution of the following linear programming problem such that:

	$max (z; α; β; γ) \in R^{(n + 2) m + 1} z - C \sum ξ \in P \cup Q ⎛ ⎝ α_{ξ} + β_{ξ} + \sum l \neq i (ξ), j (ξ) γ_{ξ, l} ⎞ ⎠$	(12)
$s.t. \forall ξ \in P \cup Q,$	$z+\textcolorblackξj(ξ)+ωj(ξ)\textcolorblack−ξi(ξ)−ωi(ξ)≤αξ,$	(13)
$\forall ξ \in P \cup Q,$	$ω_{j (ξ)} - ω_{i (ξ)} \leq ξ_{i (ξ)} - ξ_{j (ξ)} + β_{ξ},$	(14)
$\forall ξ \in P \cup Q,$	$\forall l \neq i (ξ), j (ξ), ω_{l} - ω_{j (ξ)} \leq ξ_{j (ξ)} - ξ_{l} + γ_{ξ, l},$	(15)
$\forall ξ \in P \cup Q,$	$\forall l \neq i (ξ), j (ξ), α_{ξ} \geq 0, β_{ξ} \geq 0, γ_{ξ, l} \geq 0.$	(16)

There are still many open questions we can ask in terms of tropical SVMs. In general, if we use methods to find a hard margin or soft margin tropical hyperplane developed in [19], then we have to go through exponentially many linear programming problems. However, we do not know the exact time complexity to find a tropical hard margin or soft margin tropical hyperplane for a tropical SVM.

Problem 35.

What is the time complexity of a hard or a soft margin tropical hyperplane for a tropical SVM over the tropical projective torus? Is it NP-hard?

In addition, the authors in [19] focused on tropical hyperplanes for tropical SVMs over the tropical projective torus $R^{(\frac{N}{2})} / R 1$ not over the space of ultrametrics $U_{N}$ . Again note that $U_{N}$ is an union of $N - 1$ dimensional cones over $U_{N} \subset R^{(\frac{N}{2})} / R 1$ . Thus we are interested in how $U_{N}$ and a tropical SVM over $R^{(\frac{N}{2})} / R 1$ related to each other. More specifically:

Problem 36.

Can we describe how a hard or soft margin tropical hyperplane for a tropical SVM over the tropical projective torus $R^{(\frac{N}{2})} / R 1$ separates points in the space of ultrametrics $U_{N}$ in terms of geometry?

Also we are interested in defining a tropical SVM over $U_{N}$ and developing algorithms to compute them.

Problem 37.

Define tropical hard and soft margin ”hyperplane” for tropical SVMs over $U_{N}$ . To define them can we use a tropical polytope instead of a tropical hyperplane? How can we compute them? Can we formulate as an optimization problem?

For more details, see the following papers:

Tang, Wang, and Yoshida. Tropical Support Vector Machine and its Applications to Phylogenomics. [19].

6.1.2 Tropical Linear Discriminant Analysis (LDA)

In this section we discuss tropical linear discriminant analysis (LDA). LDA is one of the classical statistical methods to classify dataset into two classes or more as the same time they reduce the dimensionality.

LDA is related to PCA in a Euclidean space and these relations are shown in Figure 6. The different between PCA and LDA is how to find the direction of a linear plane.

Figure 6: There are two categories in the response variable, red and blue. The middle picture represents PCA and the right picture shows LDA on these points.

For two classes of samples $S_{1} = {u_{1}, \dots, u_{n_{1}}}, S_{2} = {v_{1}, \dots, v_{n_{2}}} \subset R^{d}$ , the linear space for the classical LDA can be found as the optimal solution of an optimization problem such that

maxw||μ′1−μ′2||2s′21+s′22 such that μ′1=1n1∑n1i=1Proj\,(ui),μ′2=1n2∑n2i=1Proj\,(vi),s′1=∑n1i=1||Proj\,(ui)−μ′1||,s′2=∑n2i=1||Proj\,(vi)−μ′2||,where Proj\,(⋅) is a projection onto a linear plane w in Rd.

(17)

Here we use the max-plus algebra in tropical setting. Also we consider the tropical projective space for now. Let $d_{t r}$ as a tropical distance between two points in the tropical projective space $R^{d} / R 1$ . Then we can formulate the tropical linear space for tropical LDA in Equation (17) as

maxwdtr(μ′1,μ′2)−s′1−s′2 such that μ′1=argminz∈w∑n1i=1dtr(z,Proj\,(ui)),μ′2=argminz∈w∑n2i=1dtr(z,Proj\,(vi)),s′1=minz∈w∑n1i=1dtr(z,Proj\,(ui)),s′2=minz∈w∑n2i=1dtr(z,Proj\,(vi)),where Proj\,(⋅) is a projection onto a tropical % polytope w in Rd/R1.

(18)

Problem 38.

Can we define a tropical LDA over the tropical projective space? If so how can we find a tropical linear space (or tropical polytope) for a tropical LDA?

Problem 39.

Can we define a tropical LDA over the space of ultrametrics $U_{N}$ ?

6.2 Tropical Regression

For a classical multiple linear regression, with the observed data set

{(x_{1}, y_{1}), \dots (x_{n}, y_{n})}

where $x_{i} := (x_{i}^{1}, \dots, x_{i}^{e}) \in R^{e}$ and $y_{i} \in R$ , we try to find a vector $(β_{0}, β_{1}, \dots, β_{e}) \in R^{e + 1}$ such that

Y = β_{e} X_{e} + \dots + β_{1} X_{1} + β_{0} + ϵ

where $ϵ \sim N (0, σ)$ with $N (0, σ)$

is the Gaussian distribution with the mean

0

and the standard deviation

σ

Y

is a response variable, and

X_{1}, \dots, X_{e}

are explanatory variables with the smallest following value:

n \sum i = 1 {(β_{e} x_{i}^{e} + \dots + β_{1} x_{i}^{1} + β_{0} - y_{i})}_{e}^{2} .

(19)

The value in Equation 19 is called the sum of squared residuals. Thus, for a classical multiple linear regression over the Euclidean space $R^{e}$ , we try to find the linear hyperplane with the smallest sum of squared residuals.

For tropical regression over the tropical projective space, one can define a tropical regression ”polytope” as the tropical polytope with

min n \sum i = 1 {(max {β_{e} + x_{i}^{e}, \dots, β_{1} + x_{i}^{1}, β_{0}} - y_{i})}_{e}^{2} .

It has nothing done in tropical regression. Thus, it would be interesting to see how one can define them in the tropical projective space as well as the space of ultrametrics.

References

[1] M. Akian, S. Gaubert, N. Viorel, and I. Singer. Best approximation in max-plus semimodules. Linear Algebra Appl., 435:3261–3296, 2011.
[2] F. Ardila and C. J. Klivans. The bergman complex of a matroid and phylogenetic trees. journal of combinatorial theory. Series B, 96(1):38–49, 2006.
[3] B.Gärtner and M. Jaggi. Tropical support vector machines, 2006.
[4] L.J. Billera, S.P. Holmes, and K. Vogtmann. Geometry of the space of phylogenetic trees. Adv Appl Math, 27(4):733–767, 2001.
[5] Louis J. Billera, Susan P. Holmes, and Karen Vogtmann. Geometry of the Space of Phylogenetic Trees. Advances in Applied Mathematics, 27(4):733–767, 2001.
[6] G. Cohen, S. Gaubert, and J.P. Quadrat. Duality and separation theorems in idempotent semimodules. Linear Algebra Appl., 379:395–422, 2004.
[7] A. Gavryushkin and A.J. Drummond. The spaceofultrametricphylogenetictrees. Journal ofTheoreticalBiology, 403:197–208, 2016.
[8] C.J. Jardine, N. Jardine, and R. Sibson. The Structure and Construction of Taxonomic Hierarchies. Mathematical Biosciences, 1(2):173–179, 1967.
[9] M. Joswig. Essentials of tropical combinatorics, 2017.
[10] B. Lin, B. Sturmfels, X. Tang, and R. Yoshida. Convexity in tree spaces. SIAM Discrete Math, 3:2015–2038, 2017.
[11] Bo Lin, Bernd Sturmfels, Xiaoxian Tang, and Ruriko Yoshida. Convexity in Tree Spaces. SIAM Journal on Discrete Mathematics, 31(3):2015–2038, 2017.
[12] Bo Lin and Ruriko Yoshida. Tropical Fermat–Weber Points. SIAM Journal on Discrete Mathematics, 2018. To appear. Available at arXiv:1604.04674.
[13] D. Maclagan and B. Sturmfels. Introduction to Tropical Geometry, volume 161 of Graduate Studies in Mathematics. American Mathematical Society, Providence, RI, 2015.
[14] T. M. W. Nye. Principal components analysis in the space of phylogenetic trees. Ann. Stat., 39(5):2716–2739, 2011.
[15] Tom M. W. Nye. Principal Components Analysis in the Space of Phylogenetic Trees. The Annals of Statistics, 39(5):2716–2739, 2011.
[16] R. Page, R. Yoshida, and L. Zhang. Tropical principal component analysis on the space of ultrametrics, 2019.
[17] C. Semple and M. Steel. Phylogenetics, volume 24 of Oxford Lecture Series in Mathematics and its Applications. Oxford University Press, 2003.
[18] D. Speyer and B. Sturmfels. Tropical mathematics. Mathematics Magazine, 82:163–173, 2009.
[19] X. Tang, H. Wang, and R. Yoshida. Tropical support vector machines and its applications to phylogenomics, 2020.
[20] R. Yoshida, L. Zhang, and X. Zhang. Tropical principal component analysis and its application to phylogenetics. Bulletin of Mathematical Biology, 81:568–597, 2019.

Tropical Data Science

Authors

Related Research

Navigating Diverse Data Science Learning: Critical Reflections Towards Future Practice

Data Science: Challenges and Directions

A literature survey of matrix methods for data science

Performing Creativity With Computational Tools

Tropical Support Vector Machine and its Applications to Phylogenomics

Benchmarking and Optimization of Gradient Boosted Decision Tree Algorithms

Kan Extensions in Data Science and Machine Learning

This week in AI

Outline

1 Introduction

2 Data Science Overview

2.1 Basic Definitions

2.2 Unsupervised Learning

2.3 Supervised Learning

3 Phylogenetics to Phylogenomics

3.1 Phylogenetic Trees

3.1.1 Phylogenetic Tree Reconstruction

3.2 Space of Phylogenetic Trees

Example 1.

Definition 2.

Example 3.

Definition 4 (Three Point Condition).

Definition 5 (Ultrametrics).

Theorem 6 ([8]).

Example 7.

Theorem 8.

4 Basics in Tropical Geometry

Definition 9 (Tropical arithmetic operations).

Example 10.

Definition 11 (Tropical scalar multiplication and vector addition).

Example 12.

Example 13.

Example 14.

Definition 15 (Generalized Hilbert projective metric).

Example 16.

Definition 17.

Example 18.

Example 19.

5 Tropical Unsupervised Learning

5.1 Tropical Fermat Weber Points

Proposition 20.

Proposition 21.

Problem 22.

5.2 Tropical Frécet Means

Proposition 23.

Problem 24.

Problem 25.

5.3 Tropical Principal Component Analysis (PCA)

Definition 26.

Theorem 27 ([16]).

Lemma 28 ([16]).

Conjecture 29.

6 Tropical Supervised Learning

6.1 Tropical Classifications

6.1.1 Tropical support vector machine SVMs

Definition 30.

Definition 31.

Example 32.

Definition 33.

Theorem 34 ([19]).

Problem 35.

Problem 36.

Problem 37.

6.1.2 Tropical Linear Discriminant Analysis (LDA)

Problem 38.

Problem 39.

6.2 Tropical Regression

References