Multistage Mixed Precision Iterative Refinement

by   Eda Oktay, et al.

Low precision arithmetic, in particular half precision (16-bit) floating point arithmetic, is now available in commercial hardware. Using lower precision can offer significant savings in computation and communication costs with proportional savings in energy. Motivated by this, there have recently emerged a number of new iterative refinement schemes for solving linear systems Ax=b, both based on standard LU factorization and GMRES solvers, that exploit multiple different precisions. Each particular algorithm and each combination of precisions leads to different condition number-based constraints for convergence of the backward and forward errors, and each has different performance costs. Given that the user may not necessarily know the condition number of their matrix a priori, it may be difficult to select the optimal variant for their problem. In this work, we develop a three-stage mixed precision iterative refinement solver which aims to combine existing mixed precision approaches to balance performance and accuracy and improve usability. For a given combination of precisions, the algorithm begins with the least expensive approach and convergence is monitored via inexpensive computations with quantities produced during the iteration. If slow convergence or divergence is detected using particular stopping criteria, the algorithm switches to use more expensive, but more reliable GMRES-based refinement approaches. After presenting the algorithm and its details, we perform extensive numerical experiments on a variety of random dense problems and problems from real applications. Our experiments demonstrate that the theoretical constraints derived in the literature are often overly strict in practice, further motivating the need for a multistage approach.



There are no comments yet.


page 1

page 2

page 3

page 4


Accelerated Multiple Precision Direct Method and Mixed Precision Iterative Refinement on Python Programming Environment

Current Python programming environment does not have any reliable and ef...

An Integer Arithmetic-Based Sparse Linear Solver Using a GMRES Method and Iterative Refinement

In this paper, we develop a (preconditioned) GMRES solver based on integ...

Algebraic error analysis for mixed-precision multigrid solvers

This paper establishes the first theoretical framework for analyzing the...

Mixed Precision s-step Lanczos and Conjugate Gradient Algorithms

Compared to the classical Lanczos algorithm, the s-step Lanczos variant ...

Mixed-Precision analysis of Householder QR Algorithms

Although mixed precision arithmetic has recently garnered interest for t...

Digit Stability Inference for Iterative Methods Using Redundant Number Representation

In our recent work on iterative computation in hardware, we showed that ...

Precision of the ENDGame: Mixed-precision arithmetic in the iterative solver of the Unified Model

The Met Office's weather and climate simulation code the Unified Model i...

Code Repositories

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.