Robust Ranking of Equivalent Algorithms via Relative Performance

10/14/2020
by   Aravind Sankaran, et al.
0

In scientific computing, it is common that one target computation can be translated into many different sequences of library calls, each identifying an algorithm. Although mathematically equivalent, those algorithms might exhibit significant differences in terms of performance. In practice, we observe that subsets of algorithms show comparable performance characteristics. For this reason, we aim to identify and separate not one algorithm, but the subset of algorithms, that are reliably faster than the rest. One of the motivations for this setup is that it makes it then possible to select an algorithm based on additional performance metrics such as those based on energy or scalability. To this end, instead of using the usual methods of quantifying the performance of an algorithm in absolute terms, we present a measurement-based approach that assigns a relative score to the algorithms in comparison to one another. The relative performance is encoded by sorting the algorithms based on pair-wise comparisons and by ranking them into equivalence (or performance) classes, so that multiple algorithms can obtain the same rank. We show that this approach, based on relative performance, leads to robust identification of the fastest algorithms, that is, reliable identification even under noisy system conditions.

READ FULL TEXT
research
02/25/2021

Performance Comparison for Scientific Computations on the Edge via Relative Performance

In a typical Internet-of-Things setting that involves scientific applica...
research
09/08/2012

Rank Centrality: Ranking from Pair-wise Comparisons

The question of aggregating pair-wise comparisons to obtain a global ran...
research
09/07/2022

A Test for FLOPs as a Discriminant for Linear Algebra Algorithms

Linear algebra expressions, which play a central role in countless scien...
research
11/15/2022

Byzantine Spectral Ranking

We study the problem of rank aggregation where the goal is to obtain a g...
research
02/19/2015

Just Sort It! A Simple and Effective Approach to Active Preference Learning

We address the problem of learning a ranking by using adaptively chosen ...
research
06/29/2019

Most abundant isotope peaks and efficient selection on Y=X_1+X_2+... + X_m

The isotope masses and relative abundances for each element are fundamen...
research
07/11/2021

SVP-CF: Selection via Proxy for Collaborative Filtering Data

We study the practical consequences of dataset sampling strategies on th...

Please sign up or login with your details

Forgot password? Click here to reset