share
1 Introduction
Correspondence is a ubiquitous concept in the interpretation of spatial datasets. Correspondence establishes analogy, indicating objects that play similar roles with respect to some context. For example, correspondence between template and image features supports object recognition, correspondence among isobars in a weather map aids identification of pressure troughs and ridges, and, as shown in this paper, correspondence and lack thereof among level curves in datasets like Fig. 1 supports characterization of matrix properties for scientific computing applications.
This paper develops a novel qualitative analysis mechanism for extracting and utilizing correspondence in spatial data, with particular focus on model selection in applications with sparse, noisy data. We develop our mechanism in the context of experimental algorithmics
, an emerging field that designs methodologies for empirically evaluating algorithms on realistic test problems, and interpreting and generalizing the results to guide selection of appropriate mathematical software. It is the preferred method of analysis in applications where domain knowledge is imperfect and for which our understanding of the factors influencing algorithm applicability is incomplete. For instance, when solving linear systems associated with finite-difference discretization of elliptic partial differential equations (PDEs), there is little mathematical theory to guide a choice between, say, a direct solver and an iterative Krylov solver plus preconditioner. An experimental algorithmics approach is to parameterize a suitable family of problems, and mine a database of PDE “solves” to gain insight into the likely relative performance of these two approaches
[16]. Thus experimental algorithmics is data-driven and aims to empirically capture the relationship between problem characteristics and algorithm performance.
An example spectral portrait, for a matrix with eigenvalues at 1, 2, 3, and 4. Qualitative properties of the portait correspond to important characteristics of the underlying problem that must be considered with respect to algorithm selection. For instance, a level curve surrounding multiple eigenvalues indicates a level of precision beyond which the eigenvalues cannot be distinguished.
Many tasks in experimental algorithmics involve assessing the eigenstructure of a given matrix. Eigenstructure helps characterize the stability, sensitivity, and accuracy of numerical methods as well as the fundamental tractability of problems. Recently, the spectral portrait (e.g. Fig. 1) has emerged as a tool for graphically visualizing eigenstructure. A spectral portait characterizes how the eigenvalues of a matrix change as perturbations (e.g. due to numerical error) are introduced in computations involving the matrix. Level curves in a spectral portait correspond to perturbations, and the region enclosed by a level curve contains all possible eigenvalues that are equivalent with respect to perturbations of a given magnitude. Analysis of level curves with respect to a class of perturbations reveals information about the matrix (e.g. nonnormality and defective eigenvalues) and the effects of different algorithms and numerical approximations.
Experimental algorithmics thus determines high-level properties from analysis of data from low-level computational experiments; we focus here on the particular case where such properties are extracted from graphical representations and where it is necessary to minimize the computational experiments performed (owing to the cost and complexity of conducting them). We pose the extraction of high-level properties as a model selection problem and show that correspondence can be exploited to drive data collection in order to discriminate among possible models. Each step in our framework is parameterized by domain knowledge of properties such as locality, similarity, and correspondence. Therefore the framework is generic both with respect to a variety of problems in experimental algorithmics (two case studies are presented here), and to problems in other domains (e.g. connections to weather data analysis are discussed). Furthermore, the approach leads to efficient and explainable data collection motivated directly by the need to disambiguate among high-level models.
2 Qualitative Analysis of Spatial Data
Our mechanism for qualitative analysis of correspondence is based on the Spatial Aggregation Language (SAL) [4, 17] and the ambiguity-directed sampling framework [1]. SAL programs apply a set of uniform operators and data types (Fig. 2) in order to extract multi-layer geometric and topological representations of spatial data. These operators utilize domain knowledge of physical properties such as continuity and locality, specified as metrics, adjacency relations, and equivalence predicates, to uncover regions of uniformity in spatially distributed data. Ambiguity-directed sampling focuses data collection so as to clarify difficult choice points in an aggregation hierarchy. It seeks to maximize information content while minimizing the number and expense of data samples.
As an example of aggregation, consider the construction of level curves in the spectral portrait in Fig. 1. The key steps in such an analysis (after [13]) are:
-
The input field maps sample locations (e.g. on a uniform grid) to perturbation levels, representing allowable imprecision before a point becomes indistinguishable from an eigenvalue (computational details are discussed later).
-
Aggregate points in a neighborhood graph (e.g. Delaunay triangulation or regular grid), localizing computation to spatially proximate points.
-
Interpolate values at new locations from values at neighboring samples, in this case determining locations of points with perturbation level belonging to a discrete set (e.g. , , ).
-
Classify neighboring similar-enough objects into equivalence classes with an equivalence predicate, in this case testing equality of field value.111While an implementation such as marching squares might combine interpolation, aggregation, and classification, we view them as conceptually distinct operations.
-
Redescribe each equivalence class of lower-level objects as a single higher-level object, in this case abstracting connected points into curves. The curve can be represented more compactly and abstractly (e.g. with a spline) than its set of sample points.
As a consequence of redescription, the next aggregation level can treat curves as first-class objects, and aggregate, classify, and redescribe them, for example to find curves nested around a single eigenvalue. This higher-level process uses the same operators, but with different parameters specifying locality, equivalence, and abstraction. Ambiguity arises when, for example, not enough sample points are available to be confident in a curve’s location, or in the separation of two curves. Ambiguity-directed sampling then optimizes selection of new locations (e.g. near the ambiguity) for data collection in order to clarify the decision-making.
As this example illustrates, SAL and ambiguity-directed sampling provide a suitable vocabulary (e.g. distance and similarity metrics) and mechanism (bottom-up aggregation and top-down sampling) to uncover multi-level structures in spatial data sets. Successful applications include decentralized control design [2, 3] weather data analysis [13], analysis of diffusion-reaction morphogenesis [15], and identification of pockets in -dimensional space and decomposition of a field based on control influences [1].
3 Qualitative Analysis of Correspondence
Our correspondence mechanism builds on the relationship between lower-level and higher-level objects in a SAL hierarchy (refer again to Fig. 2). The mechanism has two key steps: (1) establish analogy as a relation among lower-level constituents of higher-level objects; (2) establish correspondence between higher-level objects as an abstraction of the analogy between their constituents. For example, in object recognition, analogy might match image and template features, and correspondence might abstract the analogy as a rigid-body transformation. Similarly, in level curve analysis, analogy might match sample points on neighboring curves by location and local curvature, and correspondence might abstract the match as a parameterized deformation of spline representations of the curves. The analogy between constituents is well-defined only because of the context of the higher-level objects; higher-level correspondence then captures a more global view of the local matches.
Tab. 1 outlines our correspondence mechanism. Traditional SAL operators collect and abstract groups of lower-level objects into higher-level objects, and establish pairs of higher-level objects for which correspondence is to be considered. Two pieces of domain knowledge are then applied:
- Analogy predicate
-
indicates pairs (and confidence in the pairs) of lower-level objects that are analogous with respect to the higher-level objects they comprise. Examples include testing feature values for each pair of lower-level objects, hashing indices in the higher-level objects’ local coordinate systems, or explicitly constructing a spatial relation such as a triangulation. The predicate can enforce bijective analogy if appropriate. The analogize operator in Tab. 1 applies a predicate for the constituent objects and of neighboring (by ) higher-level object pairs and ; it returns a graph labeling the confidence of each analogous object pair.
- Correspondence abstraction function
-
abstracts an analogy relation on lower-level objects into a description of higher-level object correspondence. Examples include redescribing point-wise matches as a rigid-body transformation or a parameterized deformation. The abstraction captures confidence in the overall correspondence, for example computing root-mean squared distance or a Hausdorff metric between locations or features (e.g. local curvature) of analogous objects. Correspondence optimization thus entails adjusting the underlying analogy to maximize the abstracted confidence. Finally, while nearby constituents of one object often match nearby constituents of another, discontinuity (e.g. when an outer curve envelops two inner ones in Fig. 1) is a noteworthy event detectable with the abstraction mechanism. The correspond operator in Tab. 1 performs correspondence abstraction from an analogy , applying a function to the subgraph of on the constituent objects and of neighboring (by ) higher-level objects and ; it returns a graph labeling each such higher-level object pair with the resulting correspondence spatial object.
Given lower-level objects , aggregate, classify, and redescribe them into higher-level objects . Aggregate higher-level objects into a neighborhood graph localizing potential correspondence. Apply an analogy predicate to relate constituent lower-level objects of neighboring higher-level objects. analogize where represents constituency. Apply a correspondence abstraction function to establish correspondence between higher-level objects based on the analogy on their constituent lower-level objects. correspond where obtains constituents, is the subgraph for the given nodes, and returns a spatial object () representing the abstracted correspondence.
An aggregation/correspondence hierarchy establishes a distribution of possible high-level models for an input instance, thereby posing a model selection problem
: choose the one that (e.g. in a maximum-likelihood sense) best matches the data. Our mechanism supports model selection in two key ways. (1) The operators estimate and optimize confidence in correspondence. Since correspondence implies mutual support among parts of a model, it can allow relatively high-confidence model selection even with sparse, noisy data. (2) The operators bridge the lower-/higher-level gap. This allows weaknesses and inconsistencies detected in higher-level correspondence to focus lower-level data collection to be maximally effective for model disambiguation.
4 Applications in Experimental Algorithmics
We present two case studies applying our analysis framework to experimental algorithmics problems. For each we describe the underlying numerical analysis problem, our particular solution approach, and results. To the best of our knowledge, these are the first systematic algorithms for performing complete imagistic analyses (as opposed to relying on human visual inspection [8]), and which focus data collection and evaluate models until a high-confidence model is obtained.
4.1 Matrix Spectral Portrait Analysis
Our first case study focuses on the previously introduced task of matrix spectral portrait analysis (Fig. 1). Formally, the spectral portrait of a matrix is defined as:
| (1) |
where
is the identity matrix. The singularities of this map are located at the eigenvalues of the matrix, and the analysis determines the sensitivity of computation to numerical imprecision by analyzing how the map decreases moving away from the eigenvalues. As discussed in the introduction, the region enclosed by a level curve of a certain magnitude contains all points that act as “equivalent” eigenvalues under perturbations of that magnitude. For example, Fig.
1 illustrates that, under large enough perturbation, eigenvalues at 3 and 4 are indistinguishable; after larger perturbations, their joint contour also surrounds the eigenvalue at 2, and then the eigenvalue at 1. This illustrates that sensitivity analysis is solved by identifying values at which level curves merge.Tab. 2 describes qualitative correspondence analysis of spectral portraits. Data are collected by computing Eq. 1; the analysis determines perturbation equivalence of eigenvalues by detecting curve merges via level curve correspondence. The first aggregation level generates samples on a coarse regular grid around the known eigenvalue locations, and then interpolates and abstracts level curves as in the spatial aggregation example. The second aggregation level finds correspondence among these curves from a Delaunay triangulation analogy of their constituent points. It tracks correspondence outward from the eigenvalues to establish a model of merge events, establishing confidence in such a model by good analogy (most points match) and good curve separation (regularly-spaced samples separate the two curves, providing evidence that contours do not merge at a smaller perturbation). Ambiguity-directed sampling generates additional data in order to separate curves and to ensure that each eigenvalue pair is merged at some perturbation.
Input: matrix , eigenvalues , perturbation levels . Output: such that eigenvalues and are equivalent with respect to perturbation of . Level one: Data collection: Eq. 1. Initial samples : points on coarse regular grid. Output: level curves . Aggregation: aggregate grid; interpolate points at values in ; classify by perturbation; redescribe into curves. Aggregation 2: = triangulation of points . Level two: Input: curves . Output: problem output . Aggregation: iff constituent points are neighbors in . Correspondence: Analogy: cross-curve neighbors in . Abstraction: , for % and % constituent points matched, and samples between and separated by no more than in angle around the enclosed eigenvalue. Model evaluation: follow correspondence outward from each pair of eigenvalues ; evaluate confidence with respect to . Sampling: When no for some , expand grid. When some too large, subsample on finer grid.
 |
 |
 |
 |
 |
 |
Fig. 3 demonstrates the application of the mechanism to the companion matrix of the polynomial [8] (see also Fig. 1). The initial grid has at least one sample between each eigenvalue (in the example, a resolution of 0.5) and extends one unit beyond the bounding box of the eigenvalues. While correspondence is found even with this coarse grid (left of Fig. 3), the confidence is not high (only a few samples separate the curves), and no curve surrounds all eigenvalues. Ambiguity-directed sampling computes additional points on a finer, larger grid (right of Fig. 3), yielding the correct results. We have applied this same mechanism to a variety of polynomial companion matrices with different numbers and spacings of roots; in each case, the correspondence mechanism correctly identifies the correct model with high confidence after 1-3 subsamples and 1-3 grid expansions.
4.2 Qualitative Computation of Jordan Forms
Our second case study focuses on analysis of the Jordan decomposition of a matrix. Matrix decomposition is an important technique, revealing pertinent features of a matrix and supporting algorithmic techniques in areas including data analysis, PDEs, and linear algebra. The Jordan decomposition reveals the eigenstructure of a matrix as follows. Consider a matrix that has
independent eigenvectors with eigenvalues
of multiplicity . The Jordan decomposition of contains upper triangular “blocks,” as revealed by the diagonalization:The typical approach to computing the Jordan form leads to a numerically unstable algorithm [12]; taking extra care usually requires more work than the original computation! Recently, however, an experimental algorithmics approach, inferring multiplicity from a geometric analysis of eigenvalue perturbations, has proved successful [8]. It is well known that the computed eigenvalues corresponding to the actual value are given by:
| (2) |
where is of multiplicity , and the phase of the perturbation ranges over {} if is positive and over {} if is negative. The insight of [8] is to graphically superimpose numerous such perturbed calculations so that the aggregate picture reveals eigenvalue multiplicity. The phase variations imply that computed eigenvalues lie on the vertices of a regular polygon with sides, centered on , and with diameter influenced by . For example, Fig. 4 shows perturbations for the 8-by-8 Brunet matrix with Jordan structure [8], for . The six “sticks” around the eigenvalue at 7 clearly reveal that its Jordan block is of size 3.222The multiplicity of the second eigenvalue at 7 is revealed at a smaller perturbation level. The “noise” in Fig. 4 is a consequence of having two Jordan blocks with the same eigenvalue and size, and a “ring” phenomenon studied in [10]; we do not attempt to capture these effects in this paper.
 |
Tab. 3 describes qualitative correspondence analysis of Jordan form. Data are collected by randomly perturbing at a specified magnitude ; the analysis determines multiplicity by detecting symmetry correspondence in the samples. The first aggregation level collects the samples for a given into triangles. The second aggregation level finds congruent triangles via geometric hashing [14], and uses congruence to establish analogy among triangle vertices. Correspondence abstracts the analogy into a rotation about a point (the eigenvalue), and evaluates whether each point rotates onto another and whether matches define regular polygons. Ambiguity-directed sampling collects additional random samples as necessary. A third level then compares rotations across different perturbations, re-visiting perturbations or choosing new perturbations in order to disambiguate. Fig. 5 demonstrates this mechanism on the Brunet matrix discussed above. The top part uses a small set of sample points, while the bottom two parts use a larger set and illustrate a good vs. bad correspondence.
We organized data collection into rounds of 6-8 samples each and experimented with three policies on where next to collect data after completing a round: (1) at the same perturbation level, (2) at a higher perturbation level, or (3) at the same perturbation level unless the number of posited models increased (model “hallucination,” as in bottom of Fig. 5). We tested 10 matrices across 4-10 perturbation levels each, as described in [8]. We varied a tolerance parameter for triangle congruence from 0.1 to 0.5 (effectively increasing the number of models posited) and determined the number of rounds needed to determine the Jordan form. Policy 1 required an average of 1 round at a tolerance of 0.1, up to 2.7 rounds at 0.5. Even with a large number of models proposed, additional data quickly weeded out bad models. Policy 2 fared better only for cases where policy 1 was focused on lower perturbation levels, and policy 3 was preferable only for the Brunet-type matrices. In other words, there is no real advantage to moving across perturbation levels! In retrospect, this is not surprising since our Jordan form computation treats multiple perturbations (irresp. of level) as independent estimates of eigenstructure.
Input: matrix , perturbations , region .
Output: eigenvalue and multiplicity in region
.
Level one:
Data collection: for , compute random normwise
perturbation of as , yielding .
Initial samples: At some level .
Output: triangles .
Aggregation: triangulate (for efficiency, require 2 vertices
on convex hull).
Level two:
Input: Triangles .
Output: set of rotations .
Aggregation: congruent triangles by geom. hashing.
Correspondence:
Analogy: by triangle congruence.
Abstraction: ,
where rotation yields same analogy, with
RMSD between and rotated, and polygon regularity , where rotation maps .
Model evaluation: confidence with respect to and ; priors
support rotations around in convex hull of and by
corresponding to “reasonable” multiplicity.
Sampling: for multiple “good” models, collect additional
random samples at perturbation .
Level three:
Input: over models ()
from chosen perturbation levels ().
Output: .
Aggregation: clustering in -space.
Model evaluation: for , take
joint probability over Sampling: for high entropy in model evaluation, add samples and
re-evaluate at outlier
 |
 |
 |
 |
 |
 |
5 Discussion
Our general mechanism for uncovering and utilizing correspondence has proved successful on challenging problems from experimental algorithmics. The mechanism leverages properties such as locality, continuity, and decomposability, which are exhibited by the applications studied here as well as many physical systems. Decomposability and locality allow us to qualify correspondence in a manner that drives data collection. Contintuity allows correspondence to obtain confidence in a model even with sparse, noisy data: consistent matches among nearby constituents mutually reinforce each other, allowing correspondence abstraction to detect and filter out inconsistent interpretations.
Correspondence can be an appropriate analysis tool for a variety of reasons. In the spectral portrait application, level curves summarize groups of eigenvalue computations, so their high-level correspondence aids characterization of the underlying computation. In the Jordan form application, however, the higher-level entity has no significance as a geometric object in numerical analysis terms, but correspondence is applicable due to the semantics of superposition. This semantics also leads to phenomena such as hallucination (given enough samples, any pattern can be found), requiring a more careful treatment of decomposition.
Our work is similar in spirit to that of [13] for weather data interpretation, and can be seen as a significant generalization, formalization, and application of techniques studied there for finding correspondence in meteorological data. Similarly, our correspondence framework captures and generalizes the computation required in object recognition, allowing the body of research developed there to be applied to a broader class of applications, such as experimental algorithmics. Compared to traditional manual analyses of graphical representations in experimental algorithmics, the algorithmic nature of our approach yields advantages such as model evaluation and targeted sampling. As with compositional modeling [11], we advocate targeted use of domain knowledge, and as with qualitative/quantitative model selection (e.g. [7]), we seek to determine high level models for empirical data. Our focus is on problems requiring particular forms of domain knowledge to overcome sparsity and noise in spatial datasets. A possible direction of future work is to explore if the inclusion-exclusion methodology popular in grid algorithms [5] is also useful for tracking correspondence.
Our long-term goal is to study data collection policies and their relationships to qualitative model determination. The notion of estimating problem-solving performance by collecting data (and vice versa) is reminiscent of reinforcement learning
[6]and active learning
[9]. The decomposable nature of SAL computations promises to (i) support the design of efficient, hierarchical algorithms for model estimation and (ii) provide a deeper understanding of the recurring roles that correspondence plays in spatial data analysis.References
- [1] C. Bailey-Kellogg and N. Ramakrishnan. Ambiguity-directed sampling for qualitative analysis of sparse data from spatially distributed physical systems. In Proc. IJCAI, pages 43–50, 2001.
- [2] C. Bailey-Kellogg and F. Zhao. Influence-based model decomposition. In Proc. AAAI, 1999.
- [3] C. Bailey-Kellogg and F. Zhao. Influence-based model decomposition. Artificial Intelligence, 2001. Accepted, to appear.
- [4] C. Bailey-Kellogg, F. Zhao, and K. Yip. Spatial aggregation: language and applications. In Proc. AAAI, pages 517–522, 1996.
- [5] C. Bekas, E. Kokiopulou, I. Koutis, and E. Gallopoulos. Towards the effective parallel computation of matrix pseudospectra. In Proc. Int. Conf. Supercomputing, pages 260–269, 2001.
- [6] J.A. Boyan and A.W. Moore. Learning evaluation functions to improve optimization by local search. JMLR, 1:77–112, 2000.
- [7] A.C. Capelo, L. Ironi, and S. Tentoni. Automated mathematical modeling from experimental data: An application to material science. IEEE Trans. SMC, 28(3):356–370, 1998.
- [8] F. Chaitin-Chatelin and V. Frayssé. Lectures on Finite Precision Computations. SIAM Monographs, 1996.
- [9] D.A. Cohn, Z. Ghahramani, and M.I. Jordan. Active learning with statistical models. JAIR, 4:129–145, 1996.
- [10] A. Edelman and Y. Ma. Non-generic eigenvalue perturbations of Jordan blocks. Linear Algebra & Appl., 273(1–3):45–63, 1998.
- [11] B. Falkenhainer and K. Forbus. Compositional modeling: Finding the right model for the job. Artif. Intell., 51(1–3):95–143, 1991.
- [12] G.H. Golub and C.F. Van Loan. Matrix Computations. Johns Hopkins University Press, 1996. Third Edition.
- [13] X. Huang and F. Zhao. Relation-based aggregation: finding objects in large spatial datasets. In Proc. Intell. Data Anal., 1999.
- [14] Y. Lamdan and H. Wolfson. Geometric hashing: A general and efficient model-based recognition scheme. In Proc. ICCV, pages 238–249, 1988.
- [15] I. Ordóñez and F. Zhao. STA: Spatio-temporal aggregation with applications to analysis of diffusion-reaction phenomena. In Proc. AAAI, pages 517–523, 2000.
- [16] N. Ramakrishnan and C.J. Ribbens. Mining and visualizing recommendation spaces for elliptic PDEs with continuous attributes. ACM Trans. Math. Soft., 26(2):254–273, 2000.
- [17] K.M. Yip and F. Zhao. Spatial aggregation: theory and applications. JAIR, 5:1–26, 1996.