1 Introduction
Deep convolutional networks have achieved great success in the last years by presenting human and superhuman performance on many machine learning problems such as image classification, speech recognition and natural language processing (
LeCun et al. (2015)). Importantly, the data in these common tasks presents particular statistical properties and it normally rests on regular lattices (e.g. images) in Euclidean space (Bronstein et al. (2016)). Recently, more attention has been given to other highly relevant problems in which the input data belongs to nonEuclidean spaces. Such kind of data may present a graph structure when it represents, for instance, social networks, knowledge bases, brain activity, proteininteraction, 3D shapes and human body poses. Although some works found in the literature propose methods and network architectures specifically tailored to tackle graphlike input data (Bronstein et al. (2016); Bruna et al. (2013); Henaff et al. (2015); Li et al. (2015); Masci et al. (2015a, b)), in comparison with other topics in the field this one is still not vastly investigated.Another recent focus of interest of the machine learning community is in the detailed analysis of the functioning of deep networks and related algorithms (Daniely et al. (2016); Ghahramani (2015)
). The minimization of high dimensional nonconvex loss function by means of stochastic gradient descent techniques is theoretically unlikely, however the successful practical achievements suggest the contrary. The hypothesis that very deep neural nets do not suffer from local minima (
Dauphin et al. (2014)) is not completely proven (Swirszcz et al. (2016)). The already classical adversarial examples (Nguyen et al. (2015)), as well as new doubts about supposedly well understood questions, such as generalization (Zhang et al. (2016)), bring even more relevance to a better understanding of the methods.In the present work we aim to advance simultaneously in the two directions described above. To accomplish this goal we focus on the topological classification of graphs (Perozzi et al. (2014); Scarselli et al. (2009)). However, we restrict our attention to a particular subset of planar graphs constrained by a regular lattice. The reason for that is threefold: i) doing so we still touch upon the issue of real world graph structured data, such as the 2D pose of a human body (Andriluka et al. (2014); Jain et al. (2016)) or road networks (Masucci et al. (2009); Viana et al. (2013)); ii) we maintain the data in Euclidean space, allowing its processing with standard CNN architectures; iii) this particular class of graphs has various nontrivial statistical properties derived from percolation theory and conformal field theories (Cardy (2001); Langlands et al. (1994); Smirnov & Werner (2001)), allowing us to analytically compute various properties of a deep CNN proposed by the authors to tackle the problem.
Specifically, we introduce Mazetesting, a specialized version of the reachability problem in graphs (Yu & Cheng (2010)). In Mazetesting, random mazes, defined as by
binary images, are classified as solvable or unsolvable according to the existence of a path between given starting and ending points in the maze (vertices in the planar graph). Other recent works approach maze problems without framing them as graphs (
Tamar et al. (2016); Oh et al. (2017); Silver et al. (2017)). However, to do so with mazes (and maps) is a common practice in graph theory (Biggs et al. (1976); Schrijver (2012)) and in applied areas, such as robotics (Elfes (1989); Choset & Nagatani (2001)). Our Mazetesting problem enjoys a high degree of analytical tractability, thereby allowing us to gain important theoretical insights regarding the learning process. We propose a deep network to tackle the problem that consists oflayers, alternating convolutional, sigmoid, and skip operations, followed at the end by a logistic regression function. We prove that such a network can express an exact solution to this problem which we call the optimalBFS (breadthfirst search) minimum. We derive the shape of the cost function around this minimum. Quite surprisingly, we find that gradients around the minimum do not scale with
. This peculiar effect is attributed to rare events in the data.In addition, we shed light on a type of suboptimal local minima in the cost function which we dub "neglect minima". Such minima occur when the network discards some important features of the data samples, and instead develops a suboptimal strategy based on the remaining features. Minima similar in nature to the above optimalBFS and neglect minima are shown to occur in numerical training and dominate the training dynamics. Despite the fact the Mazetesting is a toy problem, we believe that its fundamental properties can be observed in real problems, as is frequently the case in natural phenomena (Schmidt & Lipson (2009)), making the presented analytical analysis of broader relevance.
Additionally important, our framework also relates to neural network architectures with augmented memory, such as Neural Turing Machines (
Graves et al. (2014)) and memory networks (Weston et al. (2014); Sukhbaatar et al. (2015)). The hotspot images (Fig. 7), used to track the state of our graph search algorithm, may be seen as an external memory. Therefore, to observe how activations spread from the starting to the ending point in the hotspot images, and to analyze errors and the landscape of the cost function (Sec. 5), is analogous to analyze how errors occur in the memory of the aforementioned architectures. This connection gets even stronger when such memory architectures are employed over graph structured data, to perform task such as natural language reasoning and graph search (Weston et al. (2015); Johnson (2017); Graves et al. (2016)). In these cases, it can be considered that their memories in fact encode graphs, as it happens in our framework. Thus, the present analysis may eventually help towards a better understanding of the cost functions of memory architectures, potentially leading to improvements of their weight initialization and optimization algorithms thereby facilitating training (Mishkin & Matas (2015)).The paper is organized as follows: Sec. 2 describes in detail the Mazetesting problem. In Sec. 3 we suggest an appropriate architecture for the problem. In Sec. 4 we describe an optimal set of weights for the proposed architecture and prove that it solves the problem exactly. In Sec. 5 we report on training experiments and describe the observed training phenomena. In Sec. 6 we provide an analytical understanding of the observed training phenomena. Finally, we conclude with a discussion and an outlook.
2 Mazetesting
Let us introduce the Mazetesting classification problem. Mazes are constructed as a random two dimensional, , black and white array of cells (
) where the probability (
) of having a black cell is given by , while the other cells are white. An additional image (), called the initial hotspot image, is provided. It defines the starting point by being zero (Off) everywhere except on a square of cells having the value 1 (On) chosen at a random position (see Fig.7). A Mazetesting sample (i.e. a maze and a hotspot image) is labelled Solvable if the ending point, defined as a square at the center of the maze, is reachable from the starting point (defined by the hotspot image) by moving horizontally or vertically along black cells. The sample is labelled Unsolvable otherwise.A maze in a Mazetesting sample has various nontrivial statistical properties which can be derived analytically based on results from percolation theory and conformal field theory (Cardy (2001); Langlands et al. (1994); Smirnov & Werner (2001)). Throughout this work we directly employ such statistical properties, however we refer the reader to the aforementioned references for further details and mathematical derivations.
At the particular value chosen for
, the problem is at the percolationthreshold which marks the phase transition between the two different connectivity properties of the maze: below
the chance of having a solvable maze decays exponentially with (the geometrical distance between the ending and starting points). Above it tends to a constant at large . Exactly at the chance of having a solvable maze decays as a power law (). We note in passing that although Mazetesting can be defined for any , only the choice leads to a computational problem whose typical complexity increases with .Mazetesting datasets can be produced very easily by generating random arrays and then analyzing their connectivity properties using breadthfirst search (BFS), whose worse case complexity is . Notably, as the system size grows larger, the chance of producing solvable mazes decay as , and so, for very large , the labels will be biased towards unsolvable. There are several ways to debias the dataset. One is to select an unbiased subset of it. Alternatively, one can gradually increase the size of the startingpoint to a startingsquare whose length scales as . Unless stated otherwise, we simply leave the dataset biased but define a normalized test error (), which is proportional to the average mislabeling rate of the dataset divided by the average probability of being solvable.
3 The architecture
Here we introduce an image classification architecture to tackle the Mazetesting problem. We frame maze samples as a subclass of planar graphs, defined as regular lattices in the Euclidean space, which can be handle by regular CNNs. Our architecture can express an exact solution to the problem and, at least for small Mazes (), it can find quite good solutions during training. Although applicable to general cases, graph oriented architectures find it difficult to handle large sparse graphs due to regularization issues (Henaff et al. (2015); Li et al. (2015)), whereas we show that our architecture can perform reasonably well in the planar subclass.
Our network, shown in Fig. (7), is a deep feedforward network with skip layers, followed by a logistic regression module. The deep part of the network consists of alternating convolutional and sigmoid layers. Each such layer () receives two images, one corresponding to the original maze () and the other is the output of the previous layer (). It performs the operation , where denotes a convolution, the convolutional kernel is , the kernel is , is a bias, and
is the sigmoid function. The logistic regression layer consists of two perceptrons (
), acting on as , where is the rasterized/flattened version of , is a matrix, andis a vector of dimension
. The logistic regression module outputs the label Solvable if and Unsolvable otherwise. The cost function we used during training was the standard negative loglikelihood.4 An optimal solution: The breadthfirst search minimum
As we next show, the architecture above can provide an exact solution to the problem by effectively forming a cellular automaton executing a breadthfirst search (BFS). A choice of parameters which achieves this is , , , , and , where is the index of which corresponds to the center of the maze.
Let us explain how the above neural network processes the image (see also Fig. 7). Initially is On only at the startingpoint. Passing through the first convolutionalsigmoid layer it outputs which will be On (i.e. have values close to one) on all black cells which neighbor the On cells as well as on the original starting point. Thus On regions spread on the black cluster which contains the original startingpoint, while white clusters and black clusters which do not contain the startingpoint remain Off (close to zero in ). The final logistic regression layer simply checks whether one of the cells at the center of the maze are On and outputs the labels accordingly.
To formalize the above we start by defining two activation thresholds, and , and refer to activations which are below as being Off and to those above as being On. The quantity is defined as the smallest of the three real solutions of the equation . Notably we previously chose as this is the critical value above which three real solutions to (rather than one) exist. For we choose .
Next, we go case by case and show that the action of the convolutionalsigmoid layer switches activations between Off and On just as a BFS would. This amounts to bounding the expression for all possibly subarrays of and subarrays of . There are thus possibilities to be examined.
Figure 2 shows the desired action of the layer on three important cases (AC). Each case depicts the maze shape around some arbitrary point , the hotspot image around before the action of the layer (), and the desired action of the layer (). Case A. Having a white cell at implies and therefore the argument of the above sigmoid is smaller than this regardless of at and around . Thus and so it is Off. As the activations of played no role, case A covers in fact different cases. Case B. Consider a black cell at , with in its vicinity all being Off (vicinity here refers to and its 4 neighbors). Here the argument is smaller or equal to , and so the activation remains Off as desired. Case B covers cases as the values of on the corners were irrelevant. Case C. Consider a black cell at with one or more On activations of in its vicinity. Here the argument is larger than . The sigmoid is then larger than implying it is On. Case C covers different cases. Since we exhausted all possible cases. Lastly it can be easily verified that given an On (Off) activation at the center of the full maze the logistic regression layer will output the label Solvable (Unsolvable).
Let us now determine the required depth for this specific architecture. The previous analysis tells us that at depth unsolvable mazes would always be labelled correctly however solvable mazes would be label correctly only if the shortestpath between the startingpoint and the center is or less. The worse case scenario thus occurs when the center of the maze and the startingpoint are connected by a one dimensional curve twisting its way along sites. Therefore, for perfect performance the network depth would have to scale as the number of sites namely . A tighter but probabilistic bound on the minimal depth can be established by borrowing various results from percolation theory. It is known, from Zhou et al. (2012), that the typical length of the shortest path () for critical percolation scales as , where is the geometrical distance and
. Moreover, it is known that the probability distribution
has a tail which falls as for (Dokholyan et al. (1999)). Consequently, the chance that at distance the shortest path is longer than , where is some small positive number, decays to zero and so, should scale as with a power slightly larger than (say ).5 Training experiments
We have performed several training experiments with our architecture on and mazes with depth and respectively, datasets of sizes , , and . Unless stated otherwise we used a batch size of 20 and a learning rate of . In the following, we split the experiments into two different groups corresponding to the related phenomena observed during training, which will the analyzed in detail in the next section.
OptimalBFS like minima. For mazes and a positive random initialization for and in the network found a solution with normalized test error performance in 3 out of the 7 different initializations (baseline test error was ). In all three successful cases the minima was a variant of the OptimalBFS minima which we refer to as the checkerboardBFS minima. It is similar to the optimalBFS but spreads the On activations from the startingpoint using a checkerboard pattern rather than a uniform one, as shown in Fig. 3. The fact that it reaches test error rather than zero is attributed to this checkerboard behavior which can occasionally miss out the exit point from the maze.
Neglect minima. Again for but allowing for negative entries in and
test error following 14 attempts and 500 epochs did not improve below
. Analyzing the weights of the network, the improvement over the baseline error () came solely from identifying the inverse correlation between many white cells near the center of the maze and the chance of being solvable. Notably, this heuristic approach completely neglects information regarding the startingpoint of the maze. For
mazes, despite trying several random initialization strategies including positive entries, dataset sizes, and learning rates, the network always settled into such a partial neglect minimum. In an unsuccessful attempt to push the weights away from such partial neglect behavior, we performed further training experiments with a biased dataset in which the maze itself was uncorrelated with the label. More accurately, marginalizing over the startingpoint there is an equal chance for both labels given any particular maze. To achieve this, a maze shape was chosen randomly and then many random locations were triedout for the startingpoint using that same maze. From these, we picked that resulted in a Solvable label and that resulted in an Unsolvable label. Maze shapes which were always Unsolvable were discarded. Both the and mazes trained on this biased dataset performed poorly and yielded test error. Interestingly they improved their cost function by settling into weights in which is large compared to while and were close to zero (order of ). We have verified that such weights imply that activations in the last layer have a negligible dependence on the startingpoint and a weak dependence on the maze shape. We thus refer to this minimum as a "total neglect minimum".6 Cost function landscape and the observed training phenomena
Here we seek an analytical understanding of the aforementioned training phenomena through the analysis of the cost function around solutions similar or equal to those the network settled into during training. Specifically we shall first study the cost function landscape around the optimalBFS minimum. As would become clearer at the end of that analysis, the optimal BFS shares many similarities with the checkerboardBFS minimum obtained during training and one thus expects a similar cost function landscape around both of these. The second phenomena analyzed below is the total neglect minimum obtained during training on the biased dataset. The total neglect minimum can be thought of as an extreme version of the partial neglect minima found for in the original dataset.
6.1 The shape of the cost function near the optimalBFS minimum
Our analysis of the cost function near the optimalBFS minimum will be based on two separate models capturing the short and long scale behavior of the network near this miminum. In the first model we approximate the network by linearizing its action around weak activations. This model would enable us to identify the density of what we call "bugs" in the network. In the second model we discretize the activation levels of the neural network into binary variables and study how the resulting cellular automaton behaves when such bugs are introduced. Figure
4 shows a numerical example of the dynamics we wish to analyze.6.1.1 Linearization around the optimalBFS minimum and the emergence of bugs
Unlike an algorithm, a neural network is an analog entity and so apriori there are no sharp distinctions between a functioning and a disfunctioning neural network. An algorithm can be debugged and the bug can be identified as happening at a particular time step. However it is unclear if one can generally pinpoint a layer and a region within where a deep neural network clearly malfunctioned. Interestingly we show that in our toy problem such pinpointing of errors can be done in a sharp fashion by identifying fast and local processes which cause an unwanted switching been Off and On activations in (see Fig. 4). We call these events bugs, as they are local, harmful, and have a sharp meaning in the algorithmic context.
Below we obtain asymptotic expressions for the chance of generating such bugs as the network weights are perturbed away from the optimalBFS minimum. The main result of this subsection, derived below, is that the density of bugs (or chance of bug per cell) scales as
(1) 
for and zero for where . Following the analysis below, we expect the same dependence to hold for generic small perturbations only with different and . We have tested this claim numerically on several other types of perturbations (including ones that break the rotational symmetry of the weights) and found that it holds.
To derive Eq. (1), we first recall the analysis in Sec. 4, initially as it is decreased has no effect but to shift (the Off activation threshold) to a higher value. However, at the critical value () the solution corresponding to vanishes (becomes complex) and the correspondence with the BFS algorithm no longer holds in general. This must not mean that all Off activations are no longer stable. Indeed, recall that in Sec. 4 the argument that a black Off cell in the vicinity of Off cells remains Off (Fig. 2, Case B) assumed a worse case scenario in which all the cells in its vicinity where both Off, black, and had the maximal activation allowed (). However, if some cells in its vicinity are white, their Off activations levels are mainly determined by the absence of the large term in the sigmoid argument and orders of magnitude smaller than . We come to the conclusion that black Off cells in the vicinity of many white cells are less prone to be spontaneously turned On than black Off cells which are part of a large cluster of black cells (see also the bug in Fig. 4). In fact using the same arguments one can show that infinitesimally below only uniform black mazes will cause the network to malfunction.
To further quantify this, consider a maze of size where the hotspot image is initially all zero and thus Off. Intuitively this hotspot image should be thought of as a subarea of a larger maze located far away from the startingpoint. In this case a functioning network must leave all activation levels below . To assess the chance of bugs we thus study the probability that the output of the final convolutionalsigmoid layer will have one or more On cells.
To this end, we find it useful to linearize the system around low activation yielding (see the Appendix for a complete derivation)
(2) 
where denotes black cells (), the sum is over the nearest neighboring black cells to , , and .
For a given maze (), Eq. (2), defines a linear Hermitian operator () with random offdiagonal matrix elements dictated by via the restriction of the offdiagonal terms to black cells. Stability of
activations is ensured if this linear operator is contracting or equivalently if all its eigenvalues are smaller than
in magnitude.Hermitian operators with local noisy entries have been studied extensively in physics, in the context of disordered systems and Anderson localization (Kramer & MacKinnon (1993)). Let us describe the main relevant results. For almost all ’s the spectrum of
consists of localized eigenfunctions (
). Any such function is centered around a random site () and decays exponentially away from that site with a decay length of which in our case would be a several cells long. Thus given with an eigenvalue , repeated actions of the convolutionalsigmoid layer will make in a vicinity of grow in size as . Thus gives the characteristic time it takes these localized eigenvalue to grow into an unwanted localized region with an On activation which we define as a bug.Our original question of determining the chance of bugs now translates into a linear algebra task: finding, , the number of eigenvalues in which are larger than in magnitude, averaged over , for a given . Since simply scales all eigenvalues one finds that is the number of eigenvalues larger than in with . Analyzing this latter operator, it is easy to show that the maximal eigenvalues occurs when has a uniform pattern on a large uniform region where the is black. Indeed if contains a black uniform true box of dimension , the maximal eigenvalue is easily shown to be . However the chance that such a uniform region exists goes as and so , where . This reasoning is rigorous as far as lower bounds on are concerned, however it turns out to capture the functional behavior of near accurately (Johri & Bhatt (2012)) which is given by , where the unknown constant captures the dependence on various microscopic details. In the Appendix we find numerically that . Following this we find where . The range of integration is chosen to includes all eigenvalues which, following a multiplication by , would be larger than .
To conclude we found the number of isolated unwanted On activations which develop on Off regions. Dividing this number by we obtain the density of bugs () near . The last technical step is thus to express in terms of . Focusing on the small region or , we find that occurs when , , and . Expanding around we find . Approximating the integral over and taking the leading scale dependence, we arrive at Eq. (1) with .
6.1.2 Effects of bugs on breadthfirst search
In this subsection we wish to understand the large scale effect of namely, its effect on the test error and the cost function. Our key results here are that
(3) 
for where (). Notably this expression is independent of . In the domain or equivalently a weak dependence on remains and
(4) 
despite its appearance it can be verified that the above right hand side is smaller than within its domain of applicability. Figure (5) shows a numerical verification of this last result (see Appendix for further details).
To derive Eqs. (3) and (4), we note that as a bug is created in a large maze, it quickly switches On the cells within the black "room" in which it was created. From this region it spreads according to BFS and turns On the entire cluster connected to the buggy room (see Fig. 4). To asses the effect this bug has on performance first note that solvable mazes would be labelled Solvable regardless of bugs however unsolvable mazes might appear solvable if a bug occurs on a cell which is connected to the center of the maze. Assuming we have an unsolvable maze, we thus ask what is the chance of it being classified as solvable.
Given a particular unsolvable maze instance (), the chance of classifying it as solvable is given by where counts the number of sites in the cluster connected to the central site (central cluster). The probability distribution of for percolation is known and given by (Cardy & Ziff (2003)), with being an order of one constant which depends on the underlying lattice. Since clusters have a fractional dimension, the maximal cluster size is . Consequently, averaged over all instances is given by , which can be easily expressed in terms of Gamma functions () (see Abramowitz (1974)). In the limit of , where its derivatives with respect to are maximal, it simplifies to
(5) 
whereas for , its behavior changes to . Notably once becomes of order one, several of the approximation we took break down.
Let us relate to the test error (). In Sec. (2) the cost function was defined as the mislabeling chance over the average chance of being solvable (). Following the above discussion the mislabelling chance is and consequently . Combining Eqs. 1 and 5 we obtain our key results, Eqs. (3, 4)
As a side, one should appreciate a potential training obstacle that has been avoided related to the fact that . Considering , if was simply proportional to , will have a sharp singularity near zero. For instance, as one reduces by a factor of , the gradients increase by . These effects are in accordance with ones intuition that a few bugs in a long algorithm will typically have a devastating effect on performance. Interestingly however, the essential singularity in , derived in the previous section, completely flattens the gradients near .
Thus the essentially singularity which comes directly from rare events in the dataset strongly regulates the test error and in a related way the cost function. However it also has a negative sideeffect concerning the robustness of generalization. Given a finite dataset the rarity of events is bounded and so having may still provide perfect performance. However when encountering a larger dataset some samples with rarer events (i.e. larger black region) would appear and the network will fail sharply on these (i.e. the wrong prediction would get a high probability). Further implications of this dependence on rare events on training and generalization errors will be studied in future work.
6.2 Cost function near a total neglect minima
To provide an explanation for this phenomena let us divide the activations of the upper layer to its startingpoint dependent and independent parts. Let denote the activations at the top layer. We expand them as a sum of two functions
(6) 
where the function and
are normalized such that their variance on the data (
and , respectively) is . Notably near the reported total neglect minima we found that . Also note that for the biased dataset the maze itself is uncorrelated with the labels and thus can be thought of as noise. Clearly any solution to the Maze testing problem requires the startingpoint dependent part () to become larger than the independent part (). We argue however that in the process of increasing the activations will have to go through an intermediate "noisy" region. In this noisy region grows in magnitude however much less than and in particular obeys . As shown in the Appendix the negative loglikelihood, a commonly used cost function, is proportional to for . Thus it penalizes random false predictions and, within a region obeying it has a minimum (global with respect to that region) when . The later being the definition of a total neglect minima.Establishing the above conjecture analytically requires several pathological cases to be examined and is left for future work. In this work we provide an argument for its typical correctness along with supporting numerics in the Appendix.
A deep convolution network with a finite kernel has a notion of distance and locality. For many parameters ranges it exhibits a typical correlation length (). That is a scale beyond which two activations are statistically independent. Clearly to solve the current problem has to grow to an order of such that information from the input reaches the output. However as gradually grows, relevant and irrelevant information is being mixed and propagated onto the final layer. While depends on information which is locally accessible at each layer (i.e. the maze shape), requires information to travel from the first layer to the last. Consequently and are expected to scale differently, as and resp. (for ). Given this one finds that as claimed.
Further numerical support of this conjecture is shown in the Appendix where an upper bound on the ratio is studied on 100 different paths leading from the total neglect miminum found during training to the checkerboardBFS minimum. In all cases there is a large region around the total neglect minimum in which .
7 Conclusions
Despite their blackbox reputation, in this work we were able to shed some light on how a particular deep CNN architecture learns to classify topological properties of graph structured data. Instead of focusing our attention on general graphs, which would correspond to data in nonEuclidean spaces, we restricted ourselves to planar graphs over regular lattices, which are still capable of modelling real world problems while being suitable to CNN architectures.
We described a toy problem of this type (Mazetesting) and showed that a simple CNN architecture can express an exact solution to this problem. Our main contribution was an asymptotic analysis of the cost function landscape near two types of minima which the network typically settles into: BFS type minima which effectively executes a breadthfirst search algorithm and poorly performing minima in which important features of the input are neglected.
Quite surprisingly, we found that near the BFS type minima gradients do not scale with , the maze size. This implies that global optimization approaches can find such minima in an average time that does not increase with . Such very moderate gradients are the result of an essential singularity in the cost function around the exact solution. This singularity in turn arises from rare statistical events in the data which act as early precursors to failure of the neural network thereby preventing a sharp and abrupt increase in the cost function.
In addition we identified an obstacle to learning whose severity scales with which we called neglect minima. These are poorly performing minima in which the network neglects some important features relevant for predicting the label. We conjectured that these occur since the gradual incorporation of these important features in the prediction requires some period in the training process in which predictions become more noisy. A ”wall of noise" then keeps the network in a poorly performing state.
It would be interesting to study how well the results and lessons learned here generalize to other tasks which require very deep architectures. These include the importance of rareevents, the essential singularities in the cost function, the localized nature of malfunctions (bugs), and neglect minima stabilized by walls of noise.
These conjectures potentially could be tested analytically, using other toy models as well as on real world problems, such as basic graph algorithms (e.g. shortestpath) (Graves et al. (2016)); textual reasoning on the bAbI dataset (Weston et al. (2015)), which can be modelled as a graph; and primitive operations in "memory" architectures (e.g. copy and sorting) (Graves et al. (2014)). More specifically the importance of rareevents can be analyzed by studying the statistics of errors on the dataset as it is perturbed away from a numerically obtained minimum. Technically one should test whether the perturbation induces an typical small deviation of the prediction on most samples in the dataset or rather a strong deviation on just a few samples. Bugs can be similarly identified by comparing the activations of the network on the numerically obtained minimum and on some small perturbation to that minimum while again looking at typical versus extreme deviations. Such an analysis can potentially lead to safer and more robust designs were the network fails typically and mildly rather than rarely and strongly.
Turning to partial neglect minima these can be identified provided one has some prior knowledge on the relevant features in the dataset. The correlations or mutual information between these features and the activations at the final layer can then be studied to detect any sign of neglect. If problems involving neglect are discovered it may be beneficial to add extra terms to the cost function which encourage more mutual information between these neglected features and the labels thereby overcoming the noise barrier and pushing the training dynamics away from such neglect minimum.
Acknowledgments
Rodrigo Andrade de Bem is a CAPES Foundation scholarship holder (Process no: 99999.013296/201302, Ministry of Education, Brazil).
References
 Abramowitz (1974) Abramowitz, Milton. Handbook of Mathematical Functions, With Formulas, Graphs, and Mathematical Tables,. Dover Publications, Incorporated, 1974. ISBN 0486612724.

Andriluka et al. (2014)
Andriluka, Mykhaylo, Pishchulin, Leonid, Gehler, Peter, and Schiele, Bernt.
2d human pose estimation: New benchmark and state of the art analysis.
In CVPR, 2014.  Biggs et al. (1976) Biggs, Norman, Lloyd, E Keith, and Wilson, Robin J. Graph Theory, 17361936. Oxford University Press, 1976.
 Bronstein et al. (2016) Bronstein, Michael M, Bruna, Joan, LeCun, Yann, Szlam, Arthur, and Vandergheynst, Pierre. Geometric deep learning: going beyond euclidean data. arXiv preprint arXiv:1611.08097, 2016.
 Bruna et al. (2013) Bruna, Joan, Zaremba, Wojciech, Szlam, Arthur, and LeCun, Yann. Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203, 2013.
 Cardy (2001) Cardy, John. Conformal invariance and percolation. arXiv preprint mathph/0103018, 2001.
 Cardy & Ziff (2003) Cardy, John and Ziff, Robert M. Exact results for the universal area distribution of clusters in percolation, ising, and potts models. Journal of statistical physics, 110(1):1–33, 2003.
 Choset & Nagatani (2001) Choset, Howie and Nagatani, Keiji. Topological simultaneous localization and mapping (slam): toward exact localization without explicit localization. ICRA, 2001.
 Daniely et al. (2016) Daniely, Amit, Frostig, Roy, and Singer, Yoram. Toward deeper understanding of neural networks: The power of initialization and a dual view on expressivity. In NIPS, 2016.
 Dauphin et al. (2014) Dauphin, Yann N, Pascanu, Razvan, Gulcehre, Caglar, Cho, Kyunghyun, Ganguli, Surya, and Bengio, Yoshua. Identifying and attacking the saddle point problem in highdimensional nonconvex optimization. In NIPS, 2014.
 Dokholyan et al. (1999) Dokholyan, Nikolay V, Buldyrev, Sergey V, Havlin, Shlomo, King, Peter R, Lee, Youngki, and Stanley, H.Eugene. Distribution of shortest paths in percolation. Physica A: Statistical Mechanics and its Applications, 266(1  4):55 – 61, 1999. ISSN 03784371.
 Elfes (1989) Elfes, Alberto. Using occupancy grids for mobile robot perception and navigation. Computer, 22(6):46–57, 1989.

Ghahramani (2015)
Ghahramani, Zoubin.
Probabilistic machine learning and artificial intelligence.
Nature, 521(7553):452–459, 2015.  Graves et al. (2014) Graves, Alex, Wayne, Greg, and Danihelka, Ivo. Neural turing machines. CoRR, abs/1410.5401, 2014.
 Graves et al. (2016) Graves, Alex, Wayne, Greg, Reynolds, Malcolm, Harley, Tim, Danihelka, Ivo, GrabskaBarwińska, Agnieszka, Colmenarejo, Sergio Gómez, Grefenstette, Edward, Ramalho, Tiago, Agapiou, John, et al. Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471–476, 2016.
 Henaff et al. (2015) Henaff, Mikael, Bruna, Joan, and LeCun, Yann. Deep convolutional networks on graphstructured data. arXiv preprint arXiv:1506.05163, 2015.
 Jain et al. (2016) Jain, Ashesh, Zamir, Amir R, Savarese, Silvio, and Saxena, Ashutosh. Structuralrnn: Deep learning on spatiotemporal graphs. In CVPR, 2016.
 Johnson (2017) Johnson, Daniel D. Learning graphical state transitions. ICLR, 2017.
 Johri & Bhatt (2012) Johri, S and Bhatt, RN. Singular behavior of eigenstates in anderson’s model of localization. Physical review letters, 109(7):076402, 2012.
 Kramer & MacKinnon (1993) Kramer, Bernhard and MacKinnon, Angus. Localization: theory and experiment. Reports on Progress in Physics, 56(12):1469, 1993.
 Langlands et al. (1994) Langlands, Robert, Pouliot, Philippe, and SaintAubin, Yvan. Conformal invariance in twodimensional percolation. arXiv preprint math/9401222, 1994.
 LeCun et al. (2015) LeCun, Yann, Bengio, Yoshua, and Hinton, Geoffrey. Deep learning. Nature, 521(7553):436–444, 2015.
 Li et al. (2015) Li, Yujia, Tarlow, Daniel, Brockschmidt, Marc, and Zemel, Richard. Gated graph sequence neural networks. arXiv preprint arXiv:1511.05493, 2015.

Masci et al. (2015a)
Masci, Jonathan, Boscaini, Davide, Bronstein, Michael, and Vandergheynst,
Pierre.
Geodesic convolutional neural networks on riemannian manifolds.
In ICCV Workshop, 2015a.  Masci et al. (2015b) Masci, Jonathan, Boscaini, Davide, Bronstein, Michael, and Vandergheynst, Pierre. Shapenet: Convolutional neural networks on noneuclidean manifolds. Technical report, 2015b.
 Masucci et al. (2009) Masucci, Adolfo Paolo, Smith, D, Crooks, A, and Batty, Michael. Random planar graphs and the london street network. The European Physical Journal BCondensed Matter and Complex Systems, 71(2):259–271, 2009.
 Mishkin & Matas (2015) Mishkin, Dmytro and Matas, Jiri. All you need is a good init. CoRR, abs/1511.06422, 2015.
 Nguyen et al. (2015) Nguyen, Anh, Yosinski, Jason, and Clune, Jeff. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In CVPR, 2015.
 Oh et al. (2017) Oh, Junhyuk, Singh, Satinder, and Lee, Honglak. Value prediction network. In NIPS, 2017.
 Perozzi et al. (2014) Perozzi, Bryan, AlRfou, Rami, and Skiena, Steven. Deepwalk: Online learning of social representations. In ACM SIGKDD, 2014.
 Scarselli et al. (2009) Scarselli, Franco, Gori, Marco, Tsoi, Ah Chung, Hagenbuchner, Markus, and Monfardini, Gabriele. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61–80, 2009.
 Schmidt & Lipson (2009) Schmidt, Michael and Lipson, Hod. Distilling freeform natural laws from experimental data. science, 324(5923):81–85, 2009.
 Schrijver (2012) Schrijver, Alexander. On the history of the shortest path problem. Doc. Math, 155, 2012.
 Silver et al. (2017) Silver, David, van Hasselt, Hado, Hessel, Matteo, Schaul, Tom, Guez, Arthur, Harley, Tim, DulacArnold, Gabriel, Reichert, David P., Rabinowitz, Neil C., Barreto, André, and Degris, Thomas. The predictron: Endtoend learning and planning. In ICML, 2017.
 Smirnov & Werner (2001) Smirnov, Stanislav and Werner, Wendelin. Critical exponents for twodimensional percolation. arXiv preprint math/0109120, 2001.
 Sukhbaatar et al. (2015) Sukhbaatar, Sainbayar, Szlam, Arthur, Weston, Jason, and Fergus, Rob. Endtoend memory networks. In NIPS, pp. 2440–2448, 2015.
 Swirszcz et al. (2016) Swirszcz, Grzegorz, Czarnecki, Wojciech Marian, and Pascanu, Razvan. Local minima in training of deep networks. arXiv preprint arXiv:1611.06310, 2016.
 Tamar et al. (2016) Tamar, Aviv, Wu, Yi, Thomas, Garrett, Levine, Sergey, and Abbeel, Pieter. Value iteration networks. In NIPS, 2016.
 Theano Development Team (2016) Theano Development Team. Theano: A Python framework for fast computation of mathematical expressions. arXiv eprints, abs/1605.02688, 2016.
 Viana et al. (2013) Viana, Matheus P, Bordin, Patricia, Barthelemy, Marc, and Strano, Emanuele. The simplicity of planar networks. Nature Scientific Reports, 3(arXiv: 1312.3788):3495, 2013.
 Weston et al. (2014) Weston, Jason, Chopra, Sumit, and Bordes, Antoine. Memory networks. CoRR, abs/1410.3916, 2014.
 Weston et al. (2015) Weston, Jason, Bordes, Antoine, Chopra, Sumit, and Mikolov, Tomas. Towards aicomplete question answering: A set of prerequisite toy tasks. CoRR, abs/1502.05698, 2015.
 Yu & Cheng (2010) Yu, Jeffrey Xu and Cheng, Jiefeng. Graph reachability queries: A survey. In Managing and Mining Graph Data, pp. 181–215. Springer, 2010.
 Zhang et al. (2016) Zhang, Chiyuan, Bengio, Samy, Hardt, Moritz, Recht, Benjamin, and Vinyals, Oriol. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530, 2016.
 Zhou et al. (2012) Zhou, Zongzheng, Yang, Ji, Deng, Youjin, and Ziff, Robert M. Shortestpath fractal dimension for percolation in two and three dimensions. Physical Review E, 86(6):061101, 2012.
Appendix A Visualization of the optimalBFS minimum
Appendix B Test error away from the optimalBFS solution
We have implemented the architecture described in the main text using Theano (Theano Development Team (2016)) and tested how changes as a function of () for mazes of sizes and depth (number of layers) . These depths are enough to keep the error rate negligible at . A slight change made compared to Mazetesting as described in the main text, is that the hotspot was fixed at a distance for all mazes. The size of the datasets was between and . We numerically obtained the normalized performance () as a function of and .
As it follows from Eq. (4) in the main text the curve, , for the and results should collapse on each other for . Figure (5) of the maintest depicts three such curves, two for , to give an impression of statistical error, and one for curve (green), along with the fit to the theory (dashed line). The fit, which involves two parameters (the proportionally constant in Eq. (4) of the main text and ) captures well the behavior over three orders of magnitude. As our results are only asymptotic, both in the sense of large and , minor discrepancies are expected.
Appendix C Linearization of the sigmoidconvolutional network around Off activation
To prepare the action of sigmoidconvolutional for linearization we find it useful to introduce the following variables on locations () with black () and white () cells
(7)  
(8)  
(9) 
Rewriting the action of the sigmoidconvolutional layer in terms of these we obtain
(10)  
where means summing over the 4 sites neighboring . Next we treating as small and Taylor expand
(11)  
where is the low (and marginally stable) solution of the equation .
Next in the consistency with our assumption that is small, we can assume , and obtain that and therefore, since we are working near it is negligible. The equation of now appears as
(12) 
where the summation of neighbor now includes only black cells and . Due to form the sigmoid function, is of the same magnitude as , and consequently the relative smallness of this terms is guaranteed as long as .
We thus obtained a linearized version for the sigmoidconvolutional network which is suitable for stability analysis. Packing and into vectors (,) the equation we obtained can be written as
(13) 
with being a symmetric matrix. Denoting by and
the left eigenvectors and eigenvalues of
, we multiply the above equation from the left with and obtain(14)  
Stability analysis on this last equation is straightforward: For , a stable solution exists given by . Furthermore as the matrix has strong disorder, are localized in space. Consequently is of the same order of magnitude as and as long as , these stable solutions are well within the linear approximation we have carried. For , there are no stable solutions.
There is an important qualitative lesson to be learned from applying these results on an important test case: A maze with only black cells. In this case it is easy to verify directly on the nonlinear sigmoidconvolutional map that a uniform solution becomes unstable exactly at . Would we find the same result within our linear approximation?
To answer the above, first note that the maximal eigenvalue of will be uniform with . Furthermore for an all black maze would be exactly zero and the linear equation becomes homogeneous. Consequently destabilization occurs exactly at and is not blurred by the inhomogeneous terms. Recall that is defined as the value at which the two lower solutions of and it also satisfies the equation . Taking a derivative of the former and putting one finds that . It is now easy to verify that even within the linear approximation destabilization occurs exactly at . The source of this agreement is the fact that vanishes for a uniform black maze.
The qualitative lesson here is thus the following: The eigenvectors of with large , are associated with large black regions in the maze. It is only on the boundaries of such regions that is nonzero. Consequently near the term projected on the largest eigenvalues can, to a good accuracy, be ignored and stability analysis can be carried on the homogeneous equation where means stability and implies a bug.
Appendix D Loglikelihood and noisy predictions
Consider an abstract classification tasks where data point are classified into two categories using a deterministic function and further assume for simplicity that the chance of is equal to
. Phrased as a conditional probability distribution
is given by while . Next we wish to compare the following family of approximations to(15) 
where is a random function, uncorrelated with , outputting the labels with equal probability. Notably at it yields while at it is simply the maximum entropy distribution.
Let us measure the loglikelihood of under for
(16)  
We thus find that reduces the loglikelihood in what can be viewed as a penalty to false confidence or noise. Assuming, as argued in the main text, that is constrained to be smaller than near , it is preferable to take both and to zero and reach the maximal entropy distribution. We note by passing that the same arguments could be easily generalized to taking real values leading again to an dependence in the cost function.
Let us relate the above notations to the ones in the main text. Clearly and . Next we recall that in the main text and multiplied the vectors function representing the depended and independent parts of . The probability estimated by the logistic regression module was given by
(17)  
which yields, to leading order in and
(18) 
where and understood as the taking the values . Consequently and are naturally identified with and respectively with and being normalization constants ensuring a variance of . While . Recall also that by construction of the dataset, the we thus obtain is uncorrelated with .
Appendix E Numerical support for the conjecture
Here we provide numerical evidence showing that in a large region around the total neglect minima found during the training of our architecture on the biased dataset (i.e. the one where marginalizing over the startingpoint yields a 50/50 chance of being solvable regardless of the maze shape).
For a given set of and parameters we fix the maze shape and study the variance of the top layer activations given different starting points. We pick the maximal of these and then average this maximal variance over different mazes. This yields our estimate of . In fact it is an upper bound on as this averagedmaxvariance may reflect wrong prediction provided that they depend on .
We then obtain an estimate of by again calculating the averagemaxvariance of the top layer however now with for all maze shapes.
Next we chose a 100 random paths parametrized by leading from the total neglect minima () for the total neglect through a random point at , and then to the checkerboardBFS minima at . The random point was placed within a hypercube of length
having the total neglect minima at its center. The path was a simple quadratic interpolation between the three point. The graph below shows the statistics of
on these 100 different paths. Notably no path even had within the hypercube. We have tried three different other lengths for the hyper cube ( and ) and arrived at the same conclusions.
Comments
There are no comments yet.