Lower Bounds for Caching with Delayed Hits

05/30/2020
by   Peter Manohar, et al.
0

Caches are a fundamental component of latency-sensitive computer systems. Recent work of [ASWB20] has initiated the study of delayed hits: a phenomenon in caches that occurs when the latency between the cache and backing store is much larger than the time between new requests. We present two results for the delayed hits caching model. (1) Competitive ratio lower bound. We prove that the competitive ratio of the algorithm in [ASWB20], and more generally of any deterministic online algorithm for delayed hits, is at least Omega(kZ), where k is the cache size and Z is the delay parameter. (2) Antimonotonicity of the delayed hits latency. Antimonotonicity is a naturally desirable property of cache latency: having a cache hit instead of a cache miss should result in lower overall latency. We prove that the latency of the delayed hits model is not antimonotone by exhibiting a scenario where having a cache hit instead of a miss results in an increase in overall latency. We additionally present a modification of the delayed hits model that makes the latency antimonotone.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/06/2020

Optimal Online Algorithms for File-Bundle Caching and Generalization to Distributed Caching

We consider a generalization of the standard cache problem called file-b...
research
05/16/2021

Intra-process Caching and Reuse of Threads

Creating and destroying threads on modern Linux systems incurs high late...
research
06/03/2023

On Optimal Caching and Model Multiplexing for Large Model Inference

Large Language Models (LLMs) and other large foundation models have achi...
research
01/29/2021

No-Regret Caching via Online Mirror Descent

We study an online caching problem in which requests can be served by a ...
research
05/28/2022

Spatial Locality and Granularity Change in Caching

Caches exploit temporal and spatial locality to allow a small memory to ...
research
04/11/2023

An Associativity Threshold Phenomenon in Set-Associative Caches

In an α-way set-associative cache, the cache is partitioned into disjoin...
research
07/13/2022

Caching with Reserves

Caching is a crucial component of many computer systems, so naturally it...

Please sign up or login with your details

Forgot password? Click here to reset