 # On guarding polygons with holes

There is an old conjecture by Shermer <cit.> that in a polygon with n vertices and h holes, ⌊n+h3⌋ vertex guards are sufficient to guard the entire polygon. The conjecture is proved for h=1 by Shermer <cit.> and Aggarwal <cit.> seperately. In this paper, we prove a theorem similar to the Shermer's conjecture for a special case where the goal is to guard the vertices of the polygon (not the entire polygon) which is equivalent to finding a dominating set for the visibility graph of the polygon. Our proof also guarantees that the selected vertex guards also cover the entire outer boundary (outer perimeter of the polygon) as well.

## Authors

##### This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

## 1 Introduction

A set of points is said to guard a polygon if, for every point in the polygon, there is some such that the line segment between and is inside the polygon.

The art gallery problem asks for the minimum number of guards that are sufficient to guard any polygon with vertices. There are numerous variations of the original problem that are also referred to as the art gallery problem. In some versions guards are restricted to the perimeter, or even to the vertices of the polygon which are called vertex guards. Some versions require only the perimeter or a subset of the perimeter to be guarded. The version in which guards must be placed on vertices and only vertices need to be guarded is equivalent to the minimum dominating set problem for the visibility graph of the polygon.

In graph theory, for a given graph with vertex set , is a dominating set for if every vertex has a neighbor in . Minimum dominating set problem is to find a dominating set such that the size of (denoted by ) is minimum among all dominating sets.

### Related results

Chvátal’s art gallery theorem [cha] states that vertex guards are always sufficient and sometimes necessary to guard a simple polygon with vertices. Later, Fisk [fisk] gave a short proof for Chvátal’s art gallery theorem.

O’Rourke [or] proved that any polygon with vertices and holes can be guarded with at most vertex guards. Note that is the total number of vertices of the polygon including the boundary and holes. But Shermer conjectured that any polygon with vertices and holes can always be guarded with vertex guards. This conjecture has been proved by Shermer [sher] and Aggarwal [aga] independently for . For , the conjecture is still open for more than 35 years. However Hoffmann, Kaufmann and Kriegel in [hof] and Bjorling-Sachs and Souvaine in [bj] proved Shermer’s conjecture for point guards (i.e. the guards can be chosen from any points inside or on the boundary of the polygon).

### Our result

In this paper, we prove that every polygon with holes has a special kind of triangulation to be specified shortly. Next by using this theorem, we prove that vertex guards are sufficient to guard the boundary of a polygon with vertices and holes. By boundary of we mean the outer perimeter of . As far as we know this version has not been studied.

## 2 Special triangulation

In this section, we present some basic definitions and a theorem in order to prove our main result. It has been proved that every polygon with (or without) holes can be triangulated and this triangulation is not always unique. In a given polygon with holes, a triangle in a triangulation of is called a special triangle if one of its edges is an edge of a hole and the apex vertex is a vertex of the polygon not on that hole (see Figure 1).

Every polygon with holes has a triangulation with a special triangle. Note that according to the definition of special triangle, one edge of a special triangle is always on a hole and its apex vertex is on the boundary or on a different hole. In the following we explain the proof of Theorem 2.

### 2.1 Proof of Theorem 2

Consider a triangulation of our polygon. If the number of holes is bigger than one, then there must exists an edge of this triangulation from a vertex of one of these holes to a vertex of the boundary of the polygon. If we make this edge into two parallel edges of very small distance from each other, we can make this hole into a part of the boundary, hence reduce the number of holes. A special triangle for this reduced polygon is also a special triangle for the original polygon. By induction on the number of holes, therefore it is enough to consider only the case when we have a polygon with one hole. Let us assume that this polygon with one hole has vertices, and the existence of a special triangle for a polygon with one hole is proved for the case when the number of vertices is less than . Note that the smallest possible is that happens when we have a triangle with a triangle as hole inside. Any triangulation for this particular polygon has a special triangle and in fact 3 special triangles. With this assumption, we can assume that non-adjacent vertices of the boundary can not see each other. Since if they do, the chord connecting them divide the boundary into two smaller parts where the hole is inside one of them. A special triangle for this smaller instance of a polygon with one hole, is also a special triangle for the original polygon. Hence a triangulation for the polygon does not have a triangle whose vertices are all on the boundary. This implies that any vertex of the boundary must see at least one vertex of the hole. Assume that we do not have a special triangle, we want to reach to a contradiction.

Let be a vertex of the boundary. According to the previous argument, it will see a vertex on the hole, say . Without loss of generality, we may assume that the ray in a counter clockwise rotational sweep, sees part of the edge of the hole.

Since a special triangle does not exist, this rotating ray will hit an obstacle that prevents it from seeing the entire edge . Let be the point on the edge obtained by rotating this ray until it hits an obstacle. This obstacle is either a vertex from the boundary or a vertex from the hole. Assume that it is a vertex from the boundary (we will discuss the second possibility shortly). Since the vertices on the boundary do not see non-adjacent vertices on the boundary, must be adjacent to . Then also sees the portion of and even more. Repeating the sweeping procedure with the ray , we will hit another obstacle, unless is a special triangle. The sequence of obstacles obtained this way can not be all the vertices of the boundary, because they keep seeing larger and larger portions of the edge and hence they are different and we only have a finite number of vertices of the boundary. So we will reach a vertex of the hole after say steps. The vertices are consecutive vertices on the boundary and the angles are all less than , since they are obtained by counter clockwise sweeps. The vertex will see a portion of the edge of the hole with the end-point , say . Since both of the edges of the hole with end-point are to the right of the ray , therefore we still need to rotate this ray counter clockwise along the edge and hence if we hit a boundary vertex obstacle, say , the angle is less than .

Notice that the obstacles encountered for a vertex of the boundary by rotating its corresponding ray counter-clockwise, can not all be among the vertices of the hole. In this case the rotating ray will make a full rotation of , and this is impossible since the maximum rotational angle that this ray can have is less than the angle of the vertex , which is definitely less than degrees.

Now since we are assuming that no special triangle exists, the process of sweeping rays and hitting obstacles will be continued forever. Producing a sequence of vertices of the boundary and holes. Since after reaching a boundary vertex, we can not get only vertices from the hole, the sequence of boundary vertices that are consecutive vertices of the polygon must come back to the starting point. Hence we go along all the vertices of the boundary, however the outer angles are all less than . This is a contradiction.

## 3 Guarding vertices with vertex guards

Now as a result of Theorem 2, we present our main theorem.

For a given polygon with vertices and holes, vertex guards are always sufficient to guard the vertices of and also the entire boundary.

###### Proof.

The proof is by induction on the number of holes. Chvátal’s theorem implies that when , vertex guards are sufficient to guard the entire polygon. Suppose that the theorem is proved for holes. Now suppose that we are given a polygon with vertices and holes. According to Theorem 2, there is a triangulation with a special triangle . Suppose that has a vertex , on the boundary or a hole and another edge, on some other hole. We split into two vertices and . So, is changed into a polygon with holes and vertices (See Figure 2). According to the induction assumption, we can choose vertex guards that guard the vertices of polygon and the boundary of . Since and , are guarded in , then all vertices of are guarded by at most vertex guards of . Also by induction the boundary of is guarded. On the other hand the boundary of is a subset of the boundary of , so the boundary of is guarded too. Note that if we have two vertex guards on and , in they are combined into one vertex guard. ∎

Note that in fact this proof, gives something slightly more. The guards will cover not only the entire outer perimeter of the polygon, but also the perimeter of holes with an exception of at most segments on them. The reason is that the segment that is the base of the special triangle that was used in the proof above is not necessarily guarded, so a similar induction on the number of holes will prove our claim.

## 4 Concluding remarks

In this paper, we have proved a theorem similar to the Shermer’s conjecture for a special case of the Art Gallery problem where we only need to guard the vertices of the polygon. The proof is based on the existence of a special triangle in the polygon. The proposed algorithm is simple and easy to implement. In future work one can possibly extend this idea for the general case.

Also for a given connected graph , it has been proved that the size of minimum dominating set of is . So if we construct a polygon with minimum number of holes such that its visibility graph is isomorphic to , our proof gives an upper bound of for the size of minimum dominating set of .