A Geometric Approach to Sample Compression

The Sample Compression Conjecture of Littlestone & Warmuth has remained unsolved for over two decades. This paper presents a systematic geometric investigation of the compression of finite maximum concept classes. Simple arrangements of hyperplanes in Hyperbolic space, and Piecewise-Linear hyperplane arrangements, are shown to represent maximum classes, generalizing the corresponding Euclidean result. A main result is that PL arrangements can be swept by a moving hyperplane to unlabeled d-compress any finite maximum class, forming a peeling scheme as conjectured by Kuzmin & Warmuth. A corollary is that some d-maximal classes cannot be embedded into any maximum class of VC dimension d+k, for any constant k. The construction of the PL sweeping involves Pachner moves on the one-inclusion graph, corresponding to moves of a hyperplane across the intersection of d other hyperplanes. This extends the well known Pachner moves for triangulations to cubical complexes.

READ FULL TEXT VIEW PDF

Authors

page 1

page 2

page 3

page 4

01/29/2014

Bounding Embeddings of VC Classes into Maximum Classes

One of the earliest conjectures in computational learning theory-the Sam...
12/05/2018

Unlabeled sample compression schemes and corner peelings for ample and maximum classes

We examine connections between combinatorial notions that arise in machi...
01/11/2022

Optimally compressing VC classes

Resolving a conjecture of Littlestone and Warmuth, we show that any conc...
05/25/2012

Measurability Aspects of the Compactness Theorem for Sample Compression Schemes

It was proved in 1998 by Ben-David and Litman that a concept space has a...
12/15/2017

Results for the maximum weight planar subgraph problem

The problem of finding the maximum-weight, planar subgraph of a finite, ...
07/14/2021

Greedy Spanners in Euclidean Spaces Admit Sublinear Separators

The greedy spanner in a low dimensional Euclidean space is a fundamental...
10/01/2020

Analysis of the weighted kappa and its maximum with Markov moves

In this paper the notion of Markov move from Algebraic Statistics is use...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Maximum concept classes have the largest cardinality possible for their given VC dimension. Such classes are of particular interest as their special recursive structure underlies all general sample compression schemes known to-date (Floyd, 1989; Warmuth, 2003; Kuzmin and Warmuth, 2007). It is this structure that admits many elegant geometric and algebraic topological representations upon which this paper focuses.

Littlestone and Warmuth (1986) introduced the study of sample compression schemes, defined as a pair of mappings for given concept class : a compression function mapping a -labeled -sample to a subsequence of labeled examples and a reconstruction function mapping the subsequence to a concept consistent with the entire -sample. A compression scheme of bounded size—the maximum cardinality of the subsequence image—was shown to imply learnability. The converse—that classes of VC dimension admit compression schemes of size —has become one of the oldest unsolved problems actively pursued within learning theory (Floyd, 1989; Helmbold et al., 1992; Ben-David and Litman, 1998; Warmuth, 2003; Hellerstein, 2006; Kuzmin and Warmuth, 2007; Rubinstein et al., 2007, 2009; Rubinstein and Rubinstein, 2008). Interest in the conjecture has been motivated by its interpretation as the converse to the existence of compression bounds for PAC learnable classes (Littlestone and Warmuth, 1986)

, the basis of practical machine learning methods on compression schemes 

(Marchand and Shawe-Taylor, 2003; von Luxburg et al., 2004), and the conjecture’s connection to a deeper understanding of the combinatorial properties of concept classes (Rubinstein et al., 2009; Rubinstein and Rubinstein, 2008). Recently Kuzmin and Warmuth (2007) achieved compression of maximum classes without the use of labels. They also conjectured that their elegant Min-Peeling Algorithm constitutes such an unlabeled -compression scheme for -maximum classes.

As discussed in our previous work (Rubinstein et al., 2009), maximum classes can be fruitfully viewed as cubical complexes. These are also topological spaces, with each cube equipped with a natural topology of open sets from its standard embedding into Euclidean space. We proved that -maximum classes correspond to -contractible complexes—topological spaces with an identity map homotopic to a constant map—extending the result that -maximum classes have trees for one-inclusion graphs. Peeling can be viewed as a special form of contractibility for maximum classes. However, there are many non-maximum contractible cubical complexes that cannot be peeled, which demonstrates that peelability reflects more detailed structure of maximum classes than given by contractibility alone.

In this paper we approach peeling from the direction of simple hyperplane arrangement representations of maximum classes. Kuzmin and Warmuth (2007, Conjecture 1) predicted that -maximum classes corresponding to simple linear hyperplane arrangements could be unlabeled -compressed by sweeping a generic hyperplane across the arrangement, and that concepts are min-peeled as their corresponding cell is swept away. We positively resolve the first part of the conjecture and show that sweeping such arrangements corresponds to a new form of corner-peeling, which we prove is distinct from min-peeling. While min-peeling removes minimum degree concepts from a one-inclusion graph, corner-peeling peels vertices that are contained in unique cubes of maximum dimension.

We explore simple hyperplane arrangements in Hyperbolic geometry, which we show correspond to a set of maximum classes, properly containing those represented by simple linear Euclidean arrangements. These classes can again be corner-peeled by sweeping. Citing the proof of existence of maximum unlabeled compression schemes due to Ben-David and Litman (1998), Kuzmin and Warmuth (2007) ask whether unlabeled compression schemes for infinite classes such as positive half spaces can be constructed explicitly. We present constructions for illustrative but simpler classes, suggesting that there are many interesting infinite maximum classes admitting explicit compression schemes, and under appropriate conditions, sweeping infinite Euclidean, Hyperbolic or PL arrangements corresponds to compression by corner-peeling.

Next we prove that all maximum classes in are represented as simple arrangements of Piecewise-Linear (PL) hyperplanes in the -ball. This extends previous work by Gärtner and Welzl (1994) on viewing simple PL hyperplane arrangements as maximum classes. The close relationship between such arrangements and their Hyperbolic versions suggests that they could be equivalent. Resolving the main problem left open in the preliminary version of this paper, (Rubinstein and Rubinstein, 2008), we show that sweeping of -contractible PL arrangements does compress all finite maximum classes by corner-peeling, completing (Kuzmin and Warmuth, 2007, Conjecture 1).

We show that a one-inclusion graph can be represented by a -contractible PL hyperplane arrangement if and only if is a strongly contractible cubical complex. This motivates the nomenclature of -contractible for the class of arrangements of PL hyperplanes. Note then that these one-inclusion graphs admit a corner-peeling scheme of the same size as the largest dimension of a cube in . Moreover if such a graph admits a corner-peeling scheme, then it is a contractible cubical complex. We give a simple example to show that there are one-inclusion graphs which admit corner-peeling schemes but are not strongly contractible and so are not represented by a -contractible PL hyperplane arrangement.

Compressing maximal classes—classes which cannot be grown without an increase to their VC dimension—is sufficient for compressing all classes, as embedded classes trivially inherit compression schemes of their super-classes. This reasoning motivates the attempt to embed -maximal classes into -maximum classes (Kuzmin and Warmuth, 2007, Open Problem 3). We present non-embeddability results following from our earlier counter-examples to Kuzmin & Warmuth’s minimum degree conjecture (Rubinstein et al., 2009), and our new results on corner-peeling. We explore with examples, maximal classes that can be compressed but not peeled, and classes that are not strongly contractible but can be compressed.

Finally, we investigate algebraic topological properties of maximum classes. Most notably we characterize -maximum classes, corresponding to simple linear Euclidean arrangements, as cubical complexes homeomorphic to the -ball. The result that such classes’ boundaries are homeomorphic to the -sphere begins the study of the boundaries of maximum classes, which are closely related to peeling. We conclude with several open problems.

2 Background

2.1 Algebraic Topology

A homeomorphism is a one-to-one and onto map between topological spaces such that both and are continuous. Spaces and are said to be homeomorphic if there exists a homeomorphism .

A homotopy is a continuous map . The initial map is restricted to and the final map is restricted to . We say that the initial and final maps are homotopic. A homotopy equivalence between spaces and is a pair of maps and such that and are homotopic to the identity maps on and respectively. We say that and have the same homotopy type if there is a homotopy equivalence between them. A deformation retraction is a special homotopy equivalence between a space and a subspace . It is a continuous map with the properties that the restriction of to is the identity map on , has range and is homotopic to the identity map on .

A cubical complex is a union of solid cubes of the form , for bounded , such that the intersection of any two cubes in the complex is either a cubical face of both cubes or the empty-set.

A contractible cubical complex is one which has the same homotopy type as a one point space . is contractible if and only if the constant map from to is a homotopy equivalence.

2.2 Concept Classes and their Learnability

A concept class on domain , is a subset of the power set of set or equivalently . We primarily consider finite domains and so will write in the sequel, where it is understood that and the dimensions or colors are identified with an ordering .

The one-inclusion graph of is the graph with vertex-set and edge-set containing iff and differ on exactly one component (Haussler et al., 1994); forms the basis of a prediction strategy with essentially-optimal worst-case expected risk. can be viewed as a simplicial complex in by filling in each face with a product of continuous intervals (Rubinstein et al., 2009). Each edge in is labeled by the component on which the two vertices differ.

A -complete collection is a union of -subcubes in , one with each choice of colors from .

Probably Approximately Correct learnability of a concept class is characterized by the finiteness of the Vapnik-Chervonenkis (VC) dimension of  (Blumer et al., 1989). One key to all such results is Sauer’s Lemma.

The VC-dimension of concept class is defined as where is the projection of on sequence .

[Vapnik and Chervonenkis, 1971; Sauer, 1972; Shelah, 1972] The cardinality of any concept classes is bounded by .

Motivated by maximizing concept class cardinality under a fixed VC-dimension, which is related to constructing general sample compression schemes (see Section 2.3), Welzl (1987) defined the following special classes.

Concept class is called maximal if for all . Furthermore if satisfies Sauer’s Lemma with equality for each , for every , then is termed maximum. If then is maximum (and hence maximal) if meets Sauer’s Lemma with equality.

The reduction of with respect to is the class where denotes the labels of the edges incident to vertex ; a multiple reduction is the result of performing several reductions in sequence. The tail of class is . Welzl showed that if is -maximum, then and are maximum of VC-dimensions and respectively.

The results presented below relate to other geometric and topological representations of maximum classes existing in the literature. Under the guise of ‘forbidden labels’, Floyd (1989) showed that maximum of VC-dim is the union of a maximally overlapping -complete collection of cubes (Rubinstein et al., 2009)—defined as a collection of -cubes which project onto all possible sets of coordinate directions. (An alternative proof was developed by Neylon 2006.) It has long been known that VC- maximum classes have one-inclusion graphs that are trees (Dudley, 1985); we previously extended this result by showing that when viewed as complexes, -maximum classes are contractible -cubical complexes (Rubinstein et al., 2009). Finally the cells of a simple linear arrangement of hyperplanes in form a VC- maximum class in the -cube (Edelsbrunner, 1987), but not all finite maximum classes correspond to such Euclidean arrangements (Floyd, 1989).

2.3 Sample Compression Schemes

Littlestone and Warmuth (1986) showed that the existence of a compression scheme of finite size is sufficient for learnability of , and conjectured the converse, that implies a compression scheme of size . Later Warmuth (2003) weakened the conjectured size to . To-date it is only known that maximum classes can be -compressed (Floyd, 1989). Unlabeled compression was first explored by Ben-David and Litman (1998); Kuzmin and Warmuth (2007) defined unlabeled compression as follows, and explicitly constructed schemes of size for maximum classes.

Let be a -maximum class on a finite domain . A representation mapping of satisfies:

  1. is a bijection between and subsets of of size at most ; and

  2.    for all , .

As with all previously published labeled schemes, all previously known unlabeled compression schemes for maximum classes exploit their special recursive projection-reduction structure and so it is doubtful that such schemes will generalize. Kuzmin and Warmuth (2007, Conjecture 2) conjectured that their Min-Peeling Algorithm constitutes an unlabeled -compression scheme for maximum classes; it iteratively removes minimum degree vertices from , representing the corresponding concepts by the remaining incident dimensions in the graph. The authors also conjectured that sweeping a hyperplane in general position across a simple linear arrangement forms a compression scheme that corresponds to min-peeling the associated maximum class (Kuzmin and Warmuth, 2007, Conjecture 1). A particularly promising approach to compressing general classes is via their maximum-embeddings: a class embedded in class trivially inherits any compression scheme for , and so an important open problem is to embed maximal classes into maximum classes with at most a linear increase in VC-dimension (Kuzmin and Warmuth, 2007, Open Problem 3).

3 Preliminaries

3.1 Constructing All Maximum Classes

The aim in this section is to describe an algorithm for constructing all maximum classes of VC dimension in the -cube. This process can be viewed as the inverse of mapping a maximum class to its -maximum projection on and the corresponding -maximum reduction.

Let be maximum classes of VC-dimensions respectively, so that , and let be -cubes, i.e., -faces of the -cube .

  1. are connected if there exists a path in the one-inclusion graph with end-points in and ; and

  2. are said to be -connected if there exists such a connecting path that further does not intersect .

The -connected components of are the equivalence classes of the -cubes of under the -connectedness relation.

The recursive algorithm for constructing all maximum classes of VC-dimension in the -cube, detailed as Algorithm 1, considers each possible -maximum class in the -cube and each possible -maximum subclass of as the projection and reduction of a -maximum class in the -cube, respectively. The algorithm lifts and to all possible maximum classes in the -cube. Then is contained in each lifted class; so all that remains is to find the tails from the complement of the reduction in the projection. It turns out that each -connected component of can be lifted to either or arbitrarily and independently of how the other -connected components are lifted. The set of lifts equates to the set of -maximum classes in the -cube that project-reduce to .

  Given:
  Returns: the set of -maximum classes in
  1.if then return  ;
  2.if then return  ;
  3. ;
      for each ,                  s.t. do
  4.    -connected components of  ;
  5.     ;
      done
  6.return ;
Algorithm 1

(cf. Algorithm 1) returns the set of maximum classes of VC-dimension in the -cube for all .

We proceed by induction on and . The base cases correspond to for which all maximum classes, enumerated as singletons in the -cube and the -cube respectively, are correctly produced by the algorithm. For the inductive step we assume that for all maximum classes of VC-dimension and in the -cube are already known by recursive calls to the algorithm. Given this, we will show that returns only -maximum classes in the -cube, and that all such classes are produced by the algorithm.

Let classes and be such that . Then is the union of a -complete collection and is the union of a -complete collection of cubes that are faces of the cubes of . Consider a concept class formed from and by Algorithm 1. The algorithm partitions into -connected components each of which is a union of -cubes. While is lifted to , some subset of the components are lifted to while the remaining components are lifted to . By definition is a -complete collection of cubes with cardinality equal to since  (Kuzmin and Warmuth, 2007). So is -maximum (Rubinstein et al., 2009, Theorem 34).

If we now consider any -maximum class , its projection on is a -maximum class and is the -maximum projection of all the -cubes in which contain color . It is thus clear that must be obtained by lifting parts of the -connected components of to the level and the remainder to the level, and to . We will now show that if the vertices of each component are not lifted to the same levels, then while the number of vertices in the lift match that of a -maximum class in the -cube, the number of edges are too few for such a maximum class. Define a lifting operator on as , where and

Consider now an edge in . By the definition of a -connected component there exists some such that either , or WLOG . In the first case is an edge in the lifted graph iff . In the second case contains four edges and in the last it contains a single edge. Furthermore, it is clear that this accounts for all edges in the lifted graph by considering the projection of an edge in the lifted product. Thus any lift other than those produced by Algorithm 1 induces strictly too few edges for a -maximum class in the -cube (cf. Kuzmin and Warmuth, 2007, Corollary 7.5).

3.2 Corner-Peeling

Kuzmin and Warmuth (2007, Conjecture 2) conjectured that their simple Min-Peeling procedure is a valid unlabeled compression scheme for maximum classes. Beginning with a concept class , Min-Peeling operates by iteratively removing a vertex of minimum-degree in to produce the peeled class . The concept class corresponding to is then represented by the dimensions of the edges incident to in , . Providing that no-clashing holds for the algorithm, the size of the min-peeling scheme is the largest degree encountered during peeling. Kuzmin and Warmuth predicted that this size is always at most for -maximum classes. We explore these questions for a related special case of peeling, where we prescribe which vertex to peel at step as follows.

We say that can be corner-peeled if there exists an ordering of the vertices of such that, for each where ,

  1. and ;

  2. There exists a unique cube of maximum dimension over all cubes in containing ;

  3. The neighbors of in satisfy ; and

  4. .

The are termed the corner vertices of respectively. If is the maximum degree of each in , then is -corner-peeled.

Note that we do not constrain the cubes to be of non-increasing dimension. It turns out that an important property of maximum classes is invariant to this kind of peeling.

We call a class shortest-path closed if for any , contains a path connecting of length .

If is shortest-path closed and is a corner vertex of , then is shortest-path closed.

Consider a shortest-path closed . Let be a corner vertex of , and denote the cube of maximum dimension in , containing , by . Consider . By assumption there exists a --path of length contained in . If is not in then is contained in the peeled product . If is in then must cross such that there is another path of the same length which avoids , and thus is shortest-path closed.

3.2.1 Corner-Peeling Implies Compression

If a maximum class can be corner-peeled then can be -unlabeled compressed.

The invariance of the shortest-path closed property under corner-peeling is key. The corner-peeling unlabeled compression scheme represents each by , the colors of the cube which is deleted from when is corner-peeled. We claim that any two vertices have non-clashing representatives. WLOG, suppose that . The class must contain a shortest --path . Let be the color of the single edge contained in that is incident to . Color appears once in , and is contained in . This implies that and that , and so . By construction, is a bijection between and all subsets of of cardinality .

If the oriented one-inclusion graph, with each edge directed away from the incident vertex represented by the edge’s color, has no cycles, then that representation’s compression scheme is termed acyclic (Floyd, 1989; Ben-David and Litman, 1998; Kuzmin and Warmuth, 2007).

All corner-peeling unlabeled compression schemes are acyclic.

We follow the proof that the Min-Peeling Algorithm is acyclic (Kuzmin and Warmuth, 2007). Let be a corner vertex ordering of . As a corner vertex is peeled, its unoriented incident edges are oriented away from . Thus all edges incident to are oriented away from and so the vertex cannot take part in any cycle. For assume is disjoint from all cycles. Then cannot be contained in a cycle, as all incoming edges into are incident to some vertex in . Thus the oriented is indeed acyclic.

3.3 Boundaries of Maximum Classes

We now turn to the geometric boundaries of maximum classes, which are closely related to corner-peeling.

The boundary of a -maximum class is defined as all the -subcubes which are the faces of a single -cube in .

Maximum classes, when viewed as cubical complexes, are analogous to soap films (an example of a minimal energy surface encountered in nature), which are obtained when a wire frame is dipped into a soap solution. Under this analogy the boundary corresponds to the wire frame and the number of -cubes can be considered the area of the soap film. An important property of the boundary of a maximum class is that all lifted reductions meet the boundary multiple times.

Every -maximum class has boundary containing at least two -cubes of every combination of colors, for all .

We use the lifting construction of Section 3.1. Let be a -maximum class and consider color . Then the reduction is an unrooted tree with at least two leaves, each of which lifts to an -colored edge in . Since the leaves are of degree in , the corresponding lifted edges belong to exactly one -cube in and so lie in . Consider now a -maximum class for , and make the inductive assumption that the projection has two of each type of -cube, and that the reduction has two of each type of -cube, in their boundaries. Pick colors . If then consider two -cubes colored by in . By the same argument as in the base case, these lift to two -colored cubes in . If then contains two -colored -cubes. For each cube, if the cube is contained in then it has two lifts one of which is contained in , otherwise its unique lift is contained in . Therefore contains at least two -colored cubes.

Having a large boundary is an important property of maximum classes that does not follow from contractibility.

Figure 1: The first steps of building the dunce hat in Example 3.3.

Take a -simplex with vertices . Glue the edges to to form a cone. Next glue the end loop to the edge . The result is a complex with a single vertex, edge and -simplex, which is classically known as the dunce hat (cf. Figure 1). It is not hard to verify that is contractible, but has no (geometric) boundary.

Although Theorem 3.3 will not be explicitly used in the sequel, we return to boundaries of maximum complexes later.

4 Euclidean Arrangements

A linear arrangement is a collection of oriented hyperplanes in . Each region or cell in the complement of the arrangement is naturally associated with a concept in ; the side of the hyperplane on which a cell falls determines the concept’s component. A simple arrangement is a linear arrangement in which any subset of planes has a unique point in common and all subsets of planes have an empty mutual intersection. Moreover any subset of planes meet in a plane of dimension . Such a collection of planes is also said to be in general position.

Many of the familiar operations on concept classes in the -cube have elegant analogues on arrangements.

  • Projection on corresponds to removing the plane;

  • The reduction is the new arrangement given by the intersection of ’s arrangement with the plane; and

  • The corresponding lifted reduction is the collection of cells in the arrangement that adjoin the plane.

A -cube in the one-inclusion graph corresponds to a collection of cells, all having a common -face, which is contained in the intersection of planes, and an edge corresponds to a pair of cells which have a common face on a single plane. The following result is due to Edelsbrunner (1987).

The concept class induced by a simple linear arrangement of planes in is -maximum.

Note that has VC-dimension at most , since general position is invariant to projection i.e., no planes are shattered. Since is the union of a -complete collection of cubes (every cell contains -intersection points in its boundary) it follows that is -maximum (Rubinstein et al., 2009).

Let be a simple linear arrangement of hyperplanes in with corresponding -maximum . The intersection of with a generic hyperplane corresponds to a -maximum class . In particular if all -intersection points of lie to one side of the generic hyperplane, then lies on the boundary of ; and is the disjoint union of two -maximum sub-classes.

The intersection of with a generic hyperplane is again a simple arrangement of hyperplanes but now in . Hence by Lemma 4 is a -maximum class in the -cube. since the adjacency relationships on the cells of the intersection are inherited from those of .

Suppose that all -intersections in lie in one half-space of the generic hyperplane. is the union of a -complete collection. We claim that each of these -cubes is a face of exactly one -cube in and is thus in . A -cube in corresponds to a line in where planes mutually intersect. The -cube is a face of a -cube in iff this line is further intersected by a plane. This occurs for exactly one plane, which is closest to the generic hyperplane along this intersection line. For once the -intersection point is reached, when following along the line away from the generic plane, a new cell is entered. This verifies the second part of the result.

Consider two parallel generic hyperplanes such that all -intersection points of lie in between them. We claim that each -cube in is in exactly one of the concept classes induced by the intersection of with and with . Consider an arbitrary -cube in . As before this cube corresponds to a region of a line formed by a mutual intersection of planes. Moreover this region is a ray, with one end-point at a -intersection. Because the ray begins at a point between the generic hyperplanes , it follows that the ray must cross exactly one of these.

Let be a simple linear arrangement of hyperplanes in and let be the corresponding -maximum class. Then considered as a cubical complex is homeomorphic to the -ball ; and considered as a -cubical complex is homeomorphic to the -sphere .

We construct a Voronoi cell decomposition corresponding to the set of -intersection points inside a very large ball in Euclidean space. By induction on , we claim that this is a cubical complex and the vertices and edges correspond to the class . By induction, on each hyperplane, the induced arrangement has a Voronoi cell decomposition which is a -cubical complex with edges and vertices matching the one-inclusion graph for the tail of corresponding to the label associated with the hyperplane. It is not hard to see that the Voronoi cell defined by a -intersection point on this hyperplane is a -cube. In fact, its -faces correspond to the Voronoi cells for , on each of the hyperplanes passing through . We also see that this -cube has a single vertex in the interior of each of the cells of the arrangement adjacent to . In this way, it follows that the vertices of this Voronoi cell decomposition are in bijective correspondence to the cells of the hyperplane arrangement. Finally the edges of the Voronoi cells pass through the faces in the hyperplanes. So these correspond bijectively to the edges of , as there is one edge for each face of the hyperplanes. Using a very large ball, containing all the -intersection points, the boundary faces become spherical cells. In fact, these form a spherical Voronoi cell decomposition, so it is easy to replace these by linear ones by taking the convex hull of their vertices. So a piecewise linear cubical complex is constructed, which has one-skeleton (graph consisting of all vertices and edges) isomorphic to the one-inclusion graph for .

Finally we want to prove that is homeomorphic to . This is quite easy by construction. For we see that is obtained by dividing up into Voronoi cells and replacing the spherical boundary cells by linear ones, using convex hulls of the boundary vertices. This process is clearly given by a homeomorphism by projection. In fact, the homeomorphism preserves the PL-structure so is a PL homeomorphism.

The following example demonstrates that not all maximum classes of VC-dimension are homeomorphic to the -ball. The key to such examples is branching.

0 0 0 0
1 0 0 0
0 1 0 0
0 0 1 0
1 0 1 0
1 1 0 0
0 1 1 0
1 0 0 1
1 1 0 1
0 1 0 1
0 1 1 1
Figure 2: A -maximum class in having a simple linear line arrangement in .

A simple linear arrangement in corresponds to points on the line—cells are simply intervals between these points and so corresponding -maximum classes are sticks. Any tree that is not a stick can therefore not be represented as a simple linear arrangement in and is also not homeomorphic to the -ball which is simply the interval .

As Kuzmin and Warmuth (2007) did previously, consider a generic hyperplane sweeping across a simple linear arrangement . begins with all -intersection points of lying in its positive half-space . The concept corresponding to cell is peeled from when , i.e., crosses the last -intersection point adjoining . At any step in the process, the result of peeling vertices from to reach , is captured by the arrangement for the appropriate .

Figure 3: The -maximum class in the -cube, enumerated in Figure 2.
Figure 4: A simple linear line arrangement corresponding to the class in Figure 2, swept by the dashed line. Each cell has a unique vertex.

Figure 2 enumerates the 11 vertices of a -maximum class in the -cube. Figures 4 and 4 display a hyperplane arrangement in Euclidean space and its Voronoi cell decomposition, corresponding to this maximum class. In this case, sweeping the vertical dashed line across the arrangement corresponds to a partial corner-peeling of the concept class with peeling sequence , , , , , . What remains is the -maximum stick .

Next we resolve the first half of (Kuzmin and Warmuth, 2007, Conjecture 1).

Any -maximum class corresponding to a simple linear arrangement can be corner-peeled by sweeping , and this process is a valid unlabeled compression scheme for of size .

We must show that as the -intersection point is crossed, there is a corner vertex of peeled away. It then follows that sweeping a generic hyperplane across corresponds to corner-peeling to a -maximum sub-class by Corollary 4. Moreover corresponds to a simple linear arrangement of hyperplanes in .

We proceed by induction on , noting that for corner-peeling is trivial. Consider as it approaches the -intersection point . The planes defining this point intersect in a simple arrangement of hyperplanes on . There is a compact cell for the arrangement on , which is a -simplex111A topological simplex—the convex hull of affinely independent points in . and shrinks to a point as passes through . We claim that the cell for the arrangement , whose intersection with is , is a corner vertex of . Consider the lines formed by intersections of of the hyperplanes, passing through . Each is a segment starting at and ending at without passing through any other -intersection points. So all faces of hyperplanes adjacent to meet in faces of . Thus, there are no edges in starting at the vertex corresponding to , except for those in the cube , which consists of all cells adjacent to in the arrangement . So corresponds to a corner vertex of the -cube in . Finally, just after the simplex is a point, is no longer in and so is corner-peeled from .

Theorem 3.2.1 completes the proof that this corner-peeling of constitutes unlabeled compression.

The sequence of cubes , removed when corner-peeling by sweeping simple linear arrangements, is of non-increasing dimension. In fact, there are cubes of dimension , then cubes of dimension , etc.

While corner-peeling and min-peeling share some properties in common, they are distinct procedures.

Consider sweeping a simple linear arrangement corresponding to a -maximum class. After all but one -intersection point has been swept, the corresponding corner-peeled class is the union of a single -cube with a -maximum stick. Min-peeling applied to would first peel a leaf, while corner-peeling must begin with the -cube.

The next result follows from our counter-examples to Kuzmin & Warmuth’s minimum degree conjecture (Rubinstein et al., 2009).

There is no constant so that all maximal classes of VC dimension can be embedded into maximum classes corresponding to simple hyperplane arrangements of dimension .

5 Hyperbolic Arrangements

We briefly discuss the Klein model of hyperbolic geometry (Ratcliffe, 1994, pg. 7). Consider the open unit ball in . Geodesics (lines of shortest length in the geometry) are given by intersections of straight lines in with the unit ball. Similarly planes of any dimension between and are given by intersections of such planes in with the unit ball. Note that such planes are completely determined by their spheres of intersection with the unit sphere , which is called the ideal boundary of hyperbolic space . Note that in the appropriate metric, the ideal boundary consists of points which are infinitely far from all points in the interior of the unit ball.

We can now see immediately that a simple hyperplane arrangement in can be described by taking a simple hyperplane arrangement in and intersecting it with the unit ball. However we require an important additional property to mimic the Euclidean case. Namely we add the constraint that every subcollection of of the hyperplanes in has mutual intersection points inside , and that no -intersection point lies in . We need this requirement to obtain that the resulting class is maximum.

A simple hyperbolic -arrangement is a collection of hyperplanes in with the property that every sub-collection of hyperplanes mutually intersect in a -dimensional hyperbolic plane, and that every sub-collection of hyperplanes mutually intersect as the empty set.

The concept class corresponding to a simple -arrangement of hyperbolic hyperplanes in is -maximum in the -cube.

The result follows by the same argument as before. Projection cannot shatter any -cube and the class is a complete union of -cubes, so is -maximum.

The key to why hyperbolic arrangements represent many new maximum classes is that they allow flexibility of choosing and independently. This is significant because the unit ball can be chosen to miss much of the intersections of the hyperplanes in Euclidean space. Note that the new maximum classes are embedded in maximum classes induced by arrangements of linear hyperplanes in Euclidean space.

A simple example is any -maximum class. It is easy to see that this can be realized in the hyperbolic plane by choosing an appropriate family of lines and the unit ball in the appropriate position. In fact, we can choose sets of pairs of points on the unit circle, which will be the intersections with our lines. So long as these pairs of points have the property that the smaller arcs of the circle between them are disjoint, the lines will not cross inside the disk and the desired -maximum class will be represented.

Corner-peeling maximum classes represented by hyperbolic hyperplane arrangements proceeds by sweeping, just as in the Euclidean case. Note first that intersections of the hyperplanes of the arrangement with the moving hyperplane appear precisely when there is a first intersection at the ideal boundary. Thus it is necessary to slightly perturb the collection of hyperplanes to ensure that only one new intersection with the moving hyperplane occurs at any time. Note also that new intersections of the sweeping hyperplane with the various lower dimensional planes of intersection between the hyperplanes appear similarly at the ideal boundary. The important claim to check is that the intersection at the ideal boundary between the moving hyperplane and a lower dimensional plane, consisting entirely of intersection points, corresponds to a corner-peeling move. We include two examples to illustrate the validity of this claim.

In the case of a -maximum class coming from disjoint lines in , a cell can disappear when the sweeping hyperplane meets a line at an ideal point. This cell is indeed a vertex of the tree, i.e., a corner-vertex.

Assume that we have a family of -planes in the unit -ball which meet in pairs in single lines, but there are no triple points of intersection, corresponding to a -maximum class. A corner-peeling move occurs when a region bounded by two half disks and an interval disappears, in the positive half space bounded by the sweeping hyperplane. Such a region can be visualized by taking a slice out of an orange. Note that the final point of contact between the hyperplane and the region is at the end of a line of intersection between two planes on the ideal boundary.

(a) (b)
Figure 5: -maximum classes in that can be represented as hyperbolic arrangements but not as Euclidean arrangements.

We next observe that sweeping by generic hyperbolic hyperplanes induces corner-peeling of the corresponding maximum class, extending Theorem 4. As the generic hyperplane sweeps across hyperbolic space, not only do swept cells correspond to corners of -cubes but also to corners of lower dimensional cubes as well. Moreover, the order of the dimensions of the cubes which are corner-peeled can be arbitrary—lower dimensional cubes may be corner-peeled before all the higher dimensional cubes are corner-peeled. This is in contrast to Euclidean sweepouts (cf. Corollary 4). Similar to Euclidean sweepouts, hyperbolic sweepouts correspond to corner-peeling and not min-peeling.

Any -maximum class corresponding to a simple hyperbolic -arrangement can be corner-peeled by sweeping with a generic hyperbolic hyperplane.

We follow the same strategy of the proof of Theorem 4. For sweeping in hyperbolic space , the generic hyperplane is initialized as tangent to . As is swept across , new intersections appear with just after meets the non-empty intersection of a subset of hyperplanes of with the ideal boundary. Each -cube in still corresponds to the cells adjacent to the intersection of hyperplanes. But now is a ()-dimensional hyperbolic hyperplane. A cell adjacent to is corner-peeled precisely when last intersects at a point of at the ideal boundary. As for simple linear arrangements, the general position of ensures that corner-peeling events never occur simultaneously. For the case , as for the simple linear arrangements just prior to the corner-peeling of , is homeomorphic to a -simplex with a missing face on the ideal boundary. And so as in the simple linear case, this -intersection point corresponds to a corner -cube. In the case , becomes a -simplex (as before) multiplied by . If , then the main difference is just before corner-peeling of , is homeomorphic to a -simplex which may be either closed (hence in the interior of ) or with a missing face on the ideal boundary. The rest of the argument remains the same, except for one important observation.

Although swept corners in hyperbolic arrangements can be of cubes of differing dimensions, these dimensions never exceed and so the proof that sweeping simple linear arrangements induces -compression schemes is still valid.

(a) (b)
Figure 6: Hyperbolic hyperplane arrangements corresponding to the classes in Figure 5. In both cases the four hyperbolic planes meet in 6 straight line segments (not shown). The planes’ colors correspond to the edges’ colors in Figure 5.

Constructed with lifting, Figure 5 completes the enumeration, up to symmetry, of the -maximum classes in begun with Example 4. These cases cannot be represented as simple Euclidean linear arrangements, since their boundaries do not satisfy the condition of Corollary 4 but can be represented as hyperbolic arrangements as in Figure 6. Figures 8 and 8 display the sweeping of a general hyperplane across the former arrangement and the corresponding corner-peeling. Notice that the corner-peeled cubes’ dimensions decrease and then increase.

Figure 7: The simple hyperbolic arrangement of Figure 6.(a) with a generic sweeping hyperplane shown in several positions before and after it sweeps past four cells.
Figure 8: The -maximum class in of Figure 5.(a), with the first four corner-vertices peeled by the hyperbolic arrangement sweeping of Figure 8. Notice that three -cubes are peeled, then a -cube (all shown) followed by -cubes.

There is no constant so that all maximal classes of VC dimension can be embedded into maximum classes corresponding to simple hyperbolic hyperplane arrangements of VC dimension .

This result follows from our counter-examples to Kuzmin & Warmuth’s minimum degree conjecture (Rubinstein et al., 2009).

Corollary 5 gives a proper superset of simple linear hyperplane arrangement-induced maximum classes as hyperbolic arrangements. We will prove in Section 7 that all maximum classes can be represented as PL hyperplane arrangements in a ball. These are the topological analogue of hyperbolic hyperplane arrangements. If the boundary of the ball is removed, then we obtain an arrangement of PL hyperplanes in Euclidean space.

6 Infinite Euclidean and Hyperbolic Arrangements

We consider a simple example of an infinite maximum class which admits corner-peeling and a compression scheme analogous to those of previous sections.

Let be the set of lines in the plane of the form and for . Let , , , and be the cells bounded by the lines , , , and , respectively. Then the cubical complex , with vertices , can be corner-peeled and hence compressed, using a sweepout by the lines for and any small fixed irrational . is a -maximum class and the unlabeled compression scheme is also of size .

To verify the properties of this example, notice that sweeping as specified corresponds to corner-peeling the vertex , then the vertices , then the remaining vertices . The lines are generic as they pass through only one intersection point of at a time. Additionally, representing by , by , by and by constitutes a valid unlabeled compression scheme. Note that the compression scheme is associated with sweeping across the arrangement in the direction of decreasing . This is necessary to pick up the boundary vertices of last in the sweepout process, so that they have either singleton representatives or the empty set. In this way, similar to Kuzmin and Warmuth (2007), we obtain a compression scheme so that every labeled sample of size is associated with a unique concept in , which is consistent with the sample. On the other hand to obtain corner-peeling, we need the sweepout to proceed with increasing so that we can begin at the boundary vertices of .

In concluding this brief discussion, we note that many infinite collections of simple hyperbolic hyperplanes and Euclidean hyperplanes can also be corner-peeled and compressed, even if intersection points and cells accumulate. However a key requirement in the Euclidean case is that the concept class has a non-empty boundary, when considered as a cubical complex. An easy approach is to assume that all the -intersections of the arrangement lie in a half-space. Moreover, since the boundary must also admit corner-peeling, we require more conditions, similar to having all the intersection points lying in an octant.

In , choose the family of planes of the form for and . A corner-peeling scheme is induced by sweeping a generic plane across the arrangement, where is a parameter and are algebraically independent (in particular, no integral linear combination is rational) and are both close to . This example has similar properties to Example 6: the compression scheme is again given by decreasing whereas corner-peeling corresponds to increasing . Note that cells shrink to points, as and the volume of cells converge to zero as , or equivalently any .

In the hyperbolic plane , represented as the unit circle centered at the origin in , choose the family of lines given by and , for . This arrangement has corner-peeling and compression schemes given by sweeping across using the generic line .

7 Piecewise-Linear Arrangements

A PL hyperplane is the image of a proper piecewise-linear homeomorphism from the -ball into , i.e., the inverse image of the boundary of the -ball is (Rourke and Sanderson, 1982). A simple PL -arrangement is an arrangement of PL hyperplanes such that every subcollection of hyperplanes meet transversely in a -dimensional PL plane for and every subcollection of hyperplanes are disjoint.

7.1 Maximum Classes are Represented by Simple PL Hyperplane Arrangements

Our aim is to prove the following theorem by a series of steps.

Every -maximum class can be represented by a simple arrangement of PL hyperplanes in an -ball. Moreover the corresponding simple arrangement of PL hyperspheres in the -sphere also represents , so long as .

7.1.1 Embedding a -Maximum Cubical Complex in the -cube into an -ball.

We begin with a -maximum cubical complex embedded into . This gives a natural embedding of into . Take a small regular neighborhood of so that the boundary of will be a closed manifold of dimension . Note that is contractible because it has a deformation retraction onto and so is a homology -sphere (by a standard, well-known argument from topology due to Mazur 1961). Our aim is to prove that is an -sphere and is an -ball. There are two ways of proving this: show that is simply connected and invoke the well-known solution to the generalised Poincaré conjecture (Smale, 1961), or use the cubical structure of the -cube and to directly prove the result. We adopt the latter approach, although the former works fine. The advantage of the latter is that it produces the required hyperplane arrangement, not just the structures of and .

7.1.2 Bisecting Sets

Figure 9: A 1-maximum class (thick solid lines) with its fattening (thin solid lines with points), bisecting sets (dashed lines) and induced complementary cells.

For each color , there is a hyperplane in

consisting of all vectors with

coordinate equal to . We can easily arrange the choice of regular neighborhood of so that is a regular neighborhood of in . (We call a bisecting set as it intersects along the ‘center’ of the reduction in the coordinate direction, see Figure 9.) But then since is a cubical complex corresponding to the reduction , by induction on , we can assert that is an -ball. Similarly the intersections can be arranged to be regular neighborhoods of -maximum classes and are also balls of dimension , etc. In this way, we see that if we can show that is an -ball, then the induction step will be satisfied and we will have produced a PL hyperplane arrangement (the system of in ) in a ball.

7.1.3 Shifting

To complete the induction step, we use the technique of shifting (Alon, 1983; Frankl, 1983; Haussler, 1995). In our situation, this can be viewed as the converse of lifting. Namely if a color is chosen, then the cubical complex has a lifted reduction consisting of all -cubes containing the color. By shifting, we can move down any of the lifted components, obtained by splitting open along , from the level to the level , to form a new cubical complex . We claim that the regular neighborhood of is a ball if and only if the same is true for . But this is quite straightforward, since the operation of shifting can be thought of as sliding components of , split open along , continuously from level to . So there is an isotopy of the attaching maps of the components onto the lifted reduction, using the product structure of the latter. It is easy then to check that this does not affect the homeomorphism type of the regular neighborhood and so the claim of shift invariance is proved.

But repeated shifting finishes with the downwards closed maximum class consisting of all vertices in the -cube with at most coordinates being one and the remaining coordinates all being zero. It is easy to see that the corresponding cubical complex is star-like, i.e., contains all the straight line segments from the origin to any point in . If we choose a regular neighborhood to also be star-like, then it is obvious that is an -ball, using radial projection. Hence our induction is complete and we have shown that any -maximum class in can be represented by a family of PL hyperplanes in the -ball.

7.1.4 Ideal Boundary

Figure 10: The top of Figure 5.(b) (i.e., the -cubes seen from above) gives part of the boundary of a regular neighborhood in .
Figure 11: The bottom of Figure 5.(b) (i.e., the -cubes seen from below) gives the rest of the boundary of a regular neighborhood.

To complete the proof of Theorem 7.1, let denote the boundary of the -ball constructed above (cf. Figures 11 and 11). Each PL hyperplane intersects this sphere in a PL hypersphere of dimension . It remains to show this arrangement of hyperspheres gives the same cubical complex as , unless .

Suppose that . Then it is easy to see that each cell in the complement of the PL hyperplane arrangement in has part of its boundary on the ideal boundary . Let , where is the intersection of with the ideal boundary and is the closure of .

It is now straightforward to verify that the face structure of is equivalent to the face structure of . Note that any family of at most PL hyperplanes meet in a PL ball properly embedded in . Since , the smallest dimension of such a ball is two, and hence its boundary is connected. Then has faces which are PL balls obtained in this way of dimension varying between and . Each of these faces has boundary a PL sphere which is a face of . So this establishes a bijection between the faces of and those of . It is easy to check that the cubical complexes corresponding to the PL hyperplanes and to the PL hyperspheres are the same.

Note that if , then any -maximum class is obtained by taking all the -faces of the -cube which contain a particular vertex. So is a -ball and the ideal boundary of is a -sphere. The cubical complex associated with the ideal boundary is the double of , i.e., two copies of glued together along their boundaries. The proof of Theorem 7.1 is now complete.

Consider the bounded below -maximum class . We claim that cannot be realized as an arrangement of PL hyperplanes in the -ball . Note that our method gives as an arrangement in and this example shows that is the best one might hope for in terms of dimension of the hyperplane arrangement.

For suppose that could be realized by any PL hyperplane arrangement in . Then clearly we can also embed into . The vertex has link given by the complete graph on vertices in . (By link, we mean the intersection of the boundary of a small ball in centered at with .) But as is well known, is not planar, i.e., cannot be embedded into the plane or -sphere. This contradiction shows that no such arrangement is possible.

7.2 Maximum Classes with Manifold Cubical Complexes

We prove a partial converse to Corollary 4: if a -maximum class has a ball as cubical complex, then it can always be realized by a simple PL hyperplane arrangement in .

Suppose that is a -maximum class. Then the following properties of , considered as a cubical complex, are equivalent:

  1. There is a simple arrangement of PL hyperplanes in which represents .

  2. is homeomorphic to the -ball.

  3. is a -manifold with boundary.

To prove (i) implies (ii), we can use exactly the same argument as Corollary 4. Next (ii) trivially implies (iii). So it remains to show that (iii) implies (i). The proof proceeds by double induction on . The initial cases where either or are very easy.

Assume that is a manifold. Let denote the coordinate projection. Then is obtained by collapsing onto , where is the reduction. As before, let be the linear hyperplane in , where the coordinate takes value . Viewing as a manifold embedded in the -cube, since intersects transversely, we see that is a proper submanifold of . But it is easy to check that collapsing to in produces a new manifold which is again homeomorphic to . (The product region in can be expanded to a larger product region and so collapsing shrinks the larger region to one of the same homeomorphism type, namely ). So we conclude that the projection is also a manifold. By induction on , it follows that there is a PL hyperplane arrangement , consisting of PL hyperplanes in , which represents .

Next, observe that the reduction can be viewed as a properly embedded submanifold in , where is a union of some of the -dimensional faces of the Voronoi cell decomposition corresponding to , described in Corollary 4. By induction on , we conclude that is also represented by PL hyperplanes in . But then since condition (i) implies (ii), it follows that is PL homeomorphic to , since the underlying cubical complex for is a -ball. So it follows that is a PL hyperplane arrangement in representing . This completes the proof that condition (iii) implies (i).

8 Corner-Peeling -Maximum Classes

We give a separate treatment for the case of -maximum classes, since it is simpler than the general case and shows by a direct geometric argument, that representation by a simple family of PL hyperplanes or PL hyperspheres implies a corner-peeling scheme.

Every -maximum class can be corner-peeled.

By Theorem 7.1, we can represent any -maximum class by a simple family of PL hyperspheres in . Every pair of hyperspheres intersects in an -sphere and there are no intersection points between any three of these hyperspheres. Consider the family of spheres