Qualitative Analysis of Correspondence for Experimental Algorithmics

Correspondence identifies relationships among objects via similarities among their components; it is ubiquitous in the analysis of spatial datasets, including images, weather maps, and computational simulations. This paper develops a novel multi-level mechanism for qualitative analysis of correspondence. Operators leverage domain knowledge to establish correspondence, evaluate implications for model selection, and leverage identified weaknesses to focus additional data collection. The utility of the mechanism is demonstrated in two applications from experimental algorithmics -- matrix spectral portrait analysis and graphical assessment of Jordan forms of matrices. Results show that the mechanism efficiently samples computational experiments and successfully uncovers high-level problem properties. It overcomes noise and data sparsity by leveraging domain knowledge to detect mutually reinforcing interpretations of spatial data.



page 9


Sampling Strategies for Mining in Data-Scarce Domains

Data mining has traditionally focused on the task of drawing inferences ...

Generalized Instrumental Variables

This paper concerns the assessment of direct causal effects from a combi...

Comments on "correspondence analysis makes you blind"

Collins' (2002) statement "correspondence analysis makes you blind" foll...

cliquematch: Finding correspondence via cliques in large graphs

The maximum clique problem finds applications in computer vision, bioinf...

A Fast and Scalable Joint Estimator for Integrating Additional Knowledge in Learning Multiple Related Sparse Gaussian Graphical Models

We consider the problem of including additional knowledge in estimating ...

Learning ODE Models with Qualitative Structure Using Gaussian Processes

Recent advances in learning techniques have enabled the modelling of dyn...

Connections between graphs and matrix spaces

Given a bipartite graph G, the graphical matrix space 𝒮_G consists of ma...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Correspondence is a ubiquitous concept in the interpretation of spatial datasets. Correspondence establishes analogy, indicating objects that play similar roles with respect to some context. For example, correspondence between template and image features supports object recognition, correspondence among isobars in a weather map aids identification of pressure troughs and ridges, and, as shown in this paper, correspondence and lack thereof among level curves in datasets like Fig. 1 supports characterization of matrix properties for scientific computing applications.

This paper develops a novel qualitative analysis mechanism for extracting and utilizing correspondence in spatial data, with particular focus on model selection in applications with sparse, noisy data. We develop our mechanism in the context of experimental algorithmics

, an emerging field that designs methodologies for empirically evaluating algorithms on realistic test problems, and interpreting and generalizing the results to guide selection of appropriate mathematical software. It is the preferred method of analysis in applications where domain knowledge is imperfect and for which our understanding of the factors influencing algorithm applicability is incomplete. For instance, when solving linear systems associated with finite-difference discretization of elliptic partial differential equations (PDEs), there is little mathematical theory to guide a choice between, say, a direct solver and an iterative Krylov solver plus preconditioner. An experimental algorithmics approach is to parameterize a suitable family of problems, and mine a database of PDE “solves” to gain insight into the likely relative performance of these two approaches 

[16]. Thus experimental algorithmics is data-driven and aims to empirically capture the relationship between problem characteristics and algorithm performance.

Figure 1:

An example spectral portrait, for a matrix with eigenvalues at 1, 2, 3, and 4. Qualitative properties of the portait correspond to important characteristics of the underlying problem that must be considered with respect to algorithm selection. For instance, a level curve surrounding multiple eigenvalues indicates a level of precision beyond which the eigenvalues cannot be distinguished.

Many tasks in experimental algorithmics involve assessing the eigenstructure of a given matrix. Eigenstructure helps characterize the stability, sensitivity, and accuracy of numerical methods as well as the fundamental tractability of problems. Recently, the spectral portrait (e.g. Fig. 1) has emerged as a tool for graphically visualizing eigenstructure. A spectral portait characterizes how the eigenvalues of a matrix change as perturbations (e.g. due to numerical error) are introduced in computations involving the matrix. Level curves in a spectral portait correspond to perturbations, and the region enclosed by a level curve contains all possible eigenvalues that are equivalent with respect to perturbations of a given magnitude. Analysis of level curves with respect to a class of perturbations reveals information about the matrix (e.g. nonnormality and defective eigenvalues) and the effects of different algorithms and numerical approximations.

Experimental algorithmics thus determines high-level properties from analysis of data from low-level computational experiments; we focus here on the particular case where such properties are extracted from graphical representations and where it is necessary to minimize the computational experiments performed (owing to the cost and complexity of conducting them). We pose the extraction of high-level properties as a model selection problem and show that correspondence can be exploited to drive data collection in order to discriminate among possible models. Each step in our framework is parameterized by domain knowledge of properties such as locality, similarity, and correspondence. Therefore the framework is generic both with respect to a variety of problems in experimental algorithmics (two case studies are presented here), and to problems in other domains (e.g. connections to weather data analysis are discussed). Furthermore, the approach leads to efficient and explainable data collection motivated directly by the need to disambiguate among high-level models.

2 Qualitative Analysis of Spatial Data

Our mechanism for qualitative analysis of correspondence is based on the Spatial Aggregation Language (SAL) [4, 17] and the ambiguity-directed sampling framework [1]. SAL programs apply a set of uniform operators and data types (Fig. 2) in order to extract multi-layer geometric and topological representations of spatial data. These operators utilize domain knowledge of physical properties such as continuity and locality, specified as metrics, adjacency relations, and equivalence predicates, to uncover regions of uniformity in spatially distributed data. Ambiguity-directed sampling focuses data collection so as to clarify difficult choice points in an aggregation hierarchy. It seeks to maximize information content while minimizing the number and expense of data samples.

Figure 2: The spatial aggregation language provides a uniform vocabulary of operators utilizing domain knowledge to build multi-layer structural descriptions of spatial data.

As an example of aggregation, consider the construction of level curves in the spectral portrait in Fig. 1. The key steps in such an analysis (after  [13]) are:

  • The input field maps sample locations (e.g. on a uniform grid) to perturbation levels, representing allowable imprecision before a point becomes indistinguishable from an eigenvalue (computational details are discussed later).

  • Aggregate points in a neighborhood graph (e.g. Delaunay triangulation or regular grid), localizing computation to spatially proximate points.

  • Interpolate values at new locations from values at neighboring samples, in this case determining locations of points with perturbation level belonging to a discrete set (e.g. , , ).

  • Classify neighboring similar-enough objects into equivalence classes with an equivalence predicate, in this case testing equality of field value.111While an implementation such as marching squares might combine interpolation, aggregation, and classification, we view them as conceptually distinct operations.

  • Redescribe each equivalence class of lower-level objects as a single higher-level object, in this case abstracting connected points into curves. The curve can be represented more compactly and abstractly (e.g. with a spline) than its set of sample points.

As a consequence of redescription, the next aggregation level can treat curves as first-class objects, and aggregate, classify, and redescribe them, for example to find curves nested around a single eigenvalue. This higher-level process uses the same operators, but with different parameters specifying locality, equivalence, and abstraction. Ambiguity arises when, for example, not enough sample points are available to be confident in a curve’s location, or in the separation of two curves. Ambiguity-directed sampling then optimizes selection of new locations (e.g. near the ambiguity) for data collection in order to clarify the decision-making.

As this example illustrates, SAL and ambiguity-directed sampling provide a suitable vocabulary (e.g. distance and similarity metrics) and mechanism (bottom-up aggregation and top-down sampling) to uncover multi-level structures in spatial data sets. Successful applications include decentralized control design [2, 3] weather data analysis [13], analysis of diffusion-reaction morphogenesis [15], and identification of pockets in -dimensional space and decomposition of a field based on control influences [1].

3 Qualitative Analysis of Correspondence

Our correspondence mechanism builds on the relationship between lower-level and higher-level objects in a SAL hierarchy (refer again to Fig. 2). The mechanism has two key steps: (1) establish analogy as a relation among lower-level constituents of higher-level objects; (2) establish correspondence between higher-level objects as an abstraction of the analogy between their constituents. For example, in object recognition, analogy might match image and template features, and correspondence might abstract the analogy as a rigid-body transformation. Similarly, in level curve analysis, analogy might match sample points on neighboring curves by location and local curvature, and correspondence might abstract the match as a parameterized deformation of spline representations of the curves. The analogy between constituents is well-defined only because of the context of the higher-level objects; higher-level correspondence then captures a more global view of the local matches.

Tab. 1 outlines our correspondence mechanism. Traditional SAL operators collect and abstract groups of lower-level objects into higher-level objects, and establish pairs of higher-level objects for which correspondence is to be considered. Two pieces of domain knowledge are then applied:

Analogy predicate

indicates pairs (and confidence in the pairs) of lower-level objects that are analogous with respect to the higher-level objects they comprise. Examples include testing feature values for each pair of lower-level objects, hashing indices in the higher-level objects’ local coordinate systems, or explicitly constructing a spatial relation such as a triangulation. The predicate can enforce bijective analogy if appropriate. The analogize operator in Tab. 1 applies a predicate for the constituent objects and of neighboring (by ) higher-level object pairs and ; it returns a graph labeling the confidence of each analogous object pair.

Correspondence abstraction function

abstracts an analogy relation on lower-level objects into a description of higher-level object correspondence. Examples include redescribing point-wise matches as a rigid-body transformation or a parameterized deformation. The abstraction captures confidence in the overall correspondence, for example computing root-mean squared distance or a Hausdorff metric between locations or features (e.g. local curvature) of analogous objects. Correspondence optimization thus entails adjusting the underlying analogy to maximize the abstracted confidence. Finally, while nearby constituents of one object often match nearby constituents of another, discontinuity (e.g. when an outer curve envelops two inner ones in Fig. 1) is a noteworthy event detectable with the abstraction mechanism. The correspond operator in Tab. 1 performs correspondence abstraction from an analogy , applying a function to the subgraph of on the constituent objects and of neighboring (by ) higher-level objects and ; it returns a graph labeling each such higher-level object pair with the resulting correspondence spatial object.

Given lower-level objects , aggregate, classify, and redescribe them into higher-level objects . Aggregate higher-level objects into a neighborhood graph localizing potential correspondence. Apply an analogy predicate to relate constituent lower-level objects of neighboring higher-level objects. analogize where represents constituency. Apply a correspondence abstraction function to establish correspondence between higher-level objects based on the analogy on their constituent lower-level objects. correspond where obtains constituents, is the subgraph for the given nodes, and returns a spatial object () representing the abstracted correspondence.

Table 1: Qualitative correspondence analysis mechanism, including formal definitions of new operators.

An aggregation/correspondence hierarchy establishes a distribution of possible high-level models for an input instance, thereby posing a model selection problem

: choose the one that (e.g. in a maximum-likelihood sense) best matches the data. Our mechanism supports model selection in two key ways. (1) The operators estimate and optimize confidence in correspondence. Since correspondence implies mutual support among parts of a model, it can allow relatively high-confidence model selection even with sparse, noisy data. (2) The operators bridge the lower-/higher-level gap. This allows weaknesses and inconsistencies detected in higher-level correspondence to focus lower-level data collection to be maximally effective for model disambiguation.

4 Applications in Experimental Algorithmics

We present two case studies applying our analysis framework to experimental algorithmics problems. For each we describe the underlying numerical analysis problem, our particular solution approach, and results. To the best of our knowledge, these are the first systematic algorithms for performing complete imagistic analyses (as opposed to relying on human visual inspection [8]), and which focus data collection and evaluate models until a high-confidence model is obtained.

4.1 Matrix Spectral Portrait Analysis

Our first case study focuses on the previously introduced task of matrix spectral portrait analysis (Fig. 1). Formally, the spectral portrait of a matrix is defined as:



is the identity matrix. The singularities of this map are located at the eigenvalues of the matrix, and the analysis determines the sensitivity of computation to numerical imprecision by analyzing how the map decreases moving away from the eigenvalues. As discussed in the introduction, the region enclosed by a level curve of a certain magnitude contains all points that act as “equivalent” eigenvalues under perturbations of that magnitude. For example, Fig. 

1 illustrates that, under large enough perturbation, eigenvalues at 3 and 4 are indistinguishable; after larger perturbations, their joint contour also surrounds the eigenvalue at 2, and then the eigenvalue at 1. This illustrates that sensitivity analysis is solved by identifying values at which level curves merge.

Tab. 2 describes qualitative correspondence analysis of spectral portraits. Data are collected by computing Eq. 1; the analysis determines perturbation equivalence of eigenvalues by detecting curve merges via level curve correspondence. The first aggregation level generates samples on a coarse regular grid around the known eigenvalue locations, and then interpolates and abstracts level curves as in the spatial aggregation example. The second aggregation level finds correspondence among these curves from a Delaunay triangulation analogy of their constituent points. It tracks correspondence outward from the eigenvalues to establish a model of merge events, establishing confidence in such a model by good analogy (most points match) and good curve separation (regularly-spaced samples separate the two curves, providing evidence that contours do not merge at a smaller perturbation). Ambiguity-directed sampling generates additional data in order to separate curves and to ensure that each eigenvalue pair is merged at some perturbation.

Input: matrix , eigenvalues , perturbation levels . Output: such that eigenvalues and are equivalent with respect to perturbation of . Level one: Data collection: Eq. 1. Initial samples : points on coarse regular grid. Output: level curves . Aggregation: aggregate grid; interpolate points at values in ; classify by perturbation; redescribe into curves. Aggregation 2: = triangulation of points . Level two: Input: curves . Output: problem output . Aggregation: iff constituent points are neighbors in . Correspondence: Analogy: cross-curve neighbors in . Abstraction: , for % and % constituent points matched, and samples between and separated by no more than in angle around the enclosed eigenvalue. Model evaluation: follow correspondence outward from each pair of eigenvalues ; evaluate confidence with respect to . Sampling: When no for some , expand grid. When some too large, subsample on finer grid.

Table 2: Correspondence mechanism instantiation for spectral portrait analysis.
Figure 3: Example of correspondence analysis of spectral portraits for (left) a small, coarse grid, and (right) an extended, subsampled grid. (top) Delaunay triangulation analogy for interpolated points comprising contours. (middle) Analogy at an example merge event; separating samples marked with “+”. (bottom) Curve-tracking tree: eigenvalues at bottom; node for curves labeled with perturbation level. Merge events indicate at what perturbation level descendant eigenvalues are indistinguishable.

Fig. 3 demonstrates the application of the mechanism to the companion matrix of the polynomial  [8] (see also Fig. 1). The initial grid has at least one sample between each eigenvalue (in the example, a resolution of 0.5) and extends one unit beyond the bounding box of the eigenvalues. While correspondence is found even with this coarse grid (left of Fig. 3), the confidence is not high (only a few samples separate the curves), and no curve surrounds all eigenvalues. Ambiguity-directed sampling computes additional points on a finer, larger grid (right of Fig. 3), yielding the correct results. We have applied this same mechanism to a variety of polynomial companion matrices with different numbers and spacings of roots; in each case, the correspondence mechanism correctly identifies the correct model with high confidence after 1-3 subsamples and 1-3 grid expansions.

4.2 Qualitative Computation of Jordan Forms

Our second case study focuses on analysis of the Jordan decomposition of a matrix. Matrix decomposition is an important technique, revealing pertinent features of a matrix and supporting algorithmic techniques in areas including data analysis, PDEs, and linear algebra. The Jordan decomposition reveals the eigenstructure of a matrix as follows. Consider a matrix that has

independent eigenvectors with eigenvalues

of multiplicity . The Jordan decomposition of contains upper triangular “blocks,” as revealed by the diagonalization:

The typical approach to computing the Jordan form leads to a numerically unstable algorithm [12]; taking extra care usually requires more work than the original computation! Recently, however, an experimental algorithmics approach, inferring multiplicity from a geometric analysis of eigenvalue perturbations, has proved successful [8]. It is well known that the computed eigenvalues corresponding to the actual value are given by:


where is of multiplicity , and the phase of the perturbation ranges over {} if is positive and over {} if is negative. The insight of [8] is to graphically superimpose numerous such perturbed calculations so that the aggregate picture reveals eigenvalue multiplicity. The phase variations imply that computed eigenvalues lie on the vertices of a regular polygon with sides, centered on , and with diameter influenced by . For example, Fig. 4 shows perturbations for the 8-by-8 Brunet matrix with Jordan structure  [8], for . The six “sticks” around the eigenvalue at 7 clearly reveal that its Jordan block is of size 3.222The multiplicity of the second eigenvalue at 7 is revealed at a smaller perturbation level. The “noise” in Fig. 4 is a consequence of having two Jordan blocks with the same eigenvalue and size, and a “ring” phenomenon studied in [10]; we do not attempt to capture these effects in this paper.

Figure 4: Superimposed spectra for assessing the Jordan form of the Brunet matrix.

Tab. 3 describes qualitative correspondence analysis of Jordan form. Data are collected by randomly perturbing at a specified magnitude ; the analysis determines multiplicity by detecting symmetry correspondence in the samples. The first aggregation level collects the samples for a given into triangles. The second aggregation level finds congruent triangles via geometric hashing [14], and uses congruence to establish analogy among triangle vertices. Correspondence abstracts the analogy into a rotation about a point (the eigenvalue), and evaluates whether each point rotates onto another and whether matches define regular polygons. Ambiguity-directed sampling collects additional random samples as necessary. A third level then compares rotations across different perturbations, re-visiting perturbations or choosing new perturbations in order to disambiguate. Fig. 5 demonstrates this mechanism on the Brunet matrix discussed above. The top part uses a small set of sample points, while the bottom two parts use a larger set and illustrate a good vs. bad correspondence.

We organized data collection into rounds of 6-8 samples each and experimented with three policies on where next to collect data after completing a round: (1) at the same perturbation level, (2) at a higher perturbation level, or (3) at the same perturbation level unless the number of posited models increased (model “hallucination,” as in bottom of Fig. 5). We tested 10 matrices across 4-10 perturbation levels each, as described in [8]. We varied a tolerance parameter for triangle congruence from 0.1 to 0.5 (effectively increasing the number of models posited) and determined the number of rounds needed to determine the Jordan form. Policy 1 required an average of 1 round at a tolerance of 0.1, up to 2.7 rounds at 0.5. Even with a large number of models proposed, additional data quickly weeded out bad models. Policy 2 fared better only for cases where policy 1 was focused on lower perturbation levels, and policy 3 was preferable only for the Brunet-type matrices. In other words, there is no real advantage to moving across perturbation levels! In retrospect, this is not surprising since our Jordan form computation treats multiple perturbations (irresp. of level) as independent estimates of eigenstructure.

Input: matrix , perturbations , region . Output: eigenvalue and multiplicity in region . Level one: Data collection: for , compute random normwise perturbation of as , yielding . Initial samples: At some level . Output: triangles . Aggregation: triangulate (for efficiency, require 2 vertices on convex hull). Level two: Input: Triangles . Output: set of rotations . Aggregation: congruent triangles by geom. hashing. Correspondence: Analogy: by triangle congruence. Abstraction: , where rotation yields same analogy, with RMSD between and rotated, and polygon regularity , where rotation maps . Model evaluation: confidence with respect to and ; priors support rotations around in convex hull of and by corresponding to “reasonable” multiplicity. Sampling: for multiple “good” models, collect additional random samples at perturbation . Level three: Input: over models () from chosen perturbation levels (). Output: . Aggregation: clustering in -space. Model evaluation: for

, take joint probability over

of .

Sampling: for high entropy in model evaluation, add samples and re-evaluate at outlier

or try new .

Table 3: Correspondence mechanism instantiation for Jordan form analysis.
Figure 5: Example of correspondence analysis of Jordan form, for (top) small sample set; (middle) larger sample set; (bottom) larger sample set but lower-scoring model. (left) Approximately-congruent triangles. (right) Evaluation of correspondence in terms of match between original (red dots) and rotated (green circles) samples.

5 Discussion

Our general mechanism for uncovering and utilizing correspondence has proved successful on challenging problems from experimental algorithmics. The mechanism leverages properties such as locality, continuity, and decomposability, which are exhibited by the applications studied here as well as many physical systems. Decomposability and locality allow us to qualify correspondence in a manner that drives data collection. Contintuity allows correspondence to obtain confidence in a model even with sparse, noisy data: consistent matches among nearby constituents mutually reinforce each other, allowing correspondence abstraction to detect and filter out inconsistent interpretations.

Correspondence can be an appropriate analysis tool for a variety of reasons. In the spectral portrait application, level curves summarize groups of eigenvalue computations, so their high-level correspondence aids characterization of the underlying computation. In the Jordan form application, however, the higher-level entity has no significance as a geometric object in numerical analysis terms, but correspondence is applicable due to the semantics of superposition. This semantics also leads to phenomena such as hallucination (given enough samples, any pattern can be found), requiring a more careful treatment of decomposition.

Our work is similar in spirit to that of [13] for weather data interpretation, and can be seen as a significant generalization, formalization, and application of techniques studied there for finding correspondence in meteorological data. Similarly, our correspondence framework captures and generalizes the computation required in object recognition, allowing the body of research developed there to be applied to a broader class of applications, such as experimental algorithmics. Compared to traditional manual analyses of graphical representations in experimental algorithmics, the algorithmic nature of our approach yields advantages such as model evaluation and targeted sampling. As with compositional modeling [11], we advocate targeted use of domain knowledge, and as with qualitative/quantitative model selection (e.g. [7]), we seek to determine high level models for empirical data. Our focus is on problems requiring particular forms of domain knowledge to overcome sparsity and noise in spatial datasets. A possible direction of future work is to explore if the inclusion-exclusion methodology popular in grid algorithms [5] is also useful for tracking correspondence.

Our long-term goal is to study data collection policies and their relationships to qualitative model determination. The notion of estimating problem-solving performance by collecting data (and vice versa) is reminiscent of reinforcement learning 


and active learning 

[9]. The decomposable nature of SAL computations promises to (i) support the design of efficient, hierarchical algorithms for model estimation and (ii) provide a deeper understanding of the recurring roles that correspondence plays in spatial data analysis.


  • [1] C. Bailey-Kellogg and N. Ramakrishnan. Ambiguity-directed sampling for qualitative analysis of sparse data from spatially distributed physical systems. In Proc. IJCAI, pages 43–50, 2001.
  • [2] C. Bailey-Kellogg and F. Zhao. Influence-based model decomposition. In Proc. AAAI, 1999.
  • [3] C. Bailey-Kellogg and F. Zhao. Influence-based model decomposition. Artificial Intelligence, 2001. Accepted, to appear.
  • [4] C. Bailey-Kellogg, F. Zhao, and K. Yip. Spatial aggregation: language and applications. In Proc. AAAI, pages 517–522, 1996.
  • [5] C. Bekas, E. Kokiopulou, I. Koutis, and E. Gallopoulos. Towards the effective parallel computation of matrix pseudospectra. In Proc. Int. Conf. Supercomputing, pages 260–269, 2001.
  • [6] J.A. Boyan and A.W. Moore. Learning evaluation functions to improve optimization by local search. JMLR, 1:77–112, 2000.
  • [7] A.C. Capelo, L. Ironi, and S. Tentoni. Automated mathematical modeling from experimental data: An application to material science. IEEE Trans. SMC, 28(3):356–370, 1998.
  • [8] F. Chaitin-Chatelin and V. Frayssé. Lectures on Finite Precision Computations. SIAM Monographs, 1996.
  • [9] D.A. Cohn, Z. Ghahramani, and M.I. Jordan. Active learning with statistical models. JAIR, 4:129–145, 1996.
  • [10] A. Edelman and Y. Ma. Non-generic eigenvalue perturbations of Jordan blocks. Linear Algebra & Appl., 273(1–3):45–63, 1998.
  • [11] B. Falkenhainer and K. Forbus. Compositional modeling: Finding the right model for the job. Artif. Intell., 51(1–3):95–143, 1991.
  • [12] G.H. Golub and C.F. Van Loan. Matrix Computations. Johns Hopkins University Press, 1996. Third Edition.
  • [13] X. Huang and F. Zhao. Relation-based aggregation: finding objects in large spatial datasets. In Proc. Intell. Data Anal., 1999.
  • [14] Y. Lamdan and H. Wolfson. Geometric hashing: A general and efficient model-based recognition scheme. In Proc. ICCV, pages 238–249, 1988.
  • [15] I. Ordóñez and F. Zhao. STA: Spatio-temporal aggregation with applications to analysis of diffusion-reaction phenomena. In Proc. AAAI, pages 517–523, 2000.
  • [16] N. Ramakrishnan and C.J. Ribbens. Mining and visualizing recommendation spaces for elliptic PDEs with continuous attributes. ACM Trans. Math. Soft., 26(2):254–273, 2000.
  • [17] K.M. Yip and F. Zhao. Spatial aggregation: theory and applications. JAIR, 5:1–26, 1996.