DeepAI
Log In Sign Up

The ERA Theorem for Safe Memory Reclamation

11/08/2022
by   Gali Sheffi, et al.
Technion
0

Safe memory reclamation (SMR) schemes for concurrent data structures offer trade-offs between three desirable properties: ease of integration, robustness, and applicability. In this paper we rigorously define SMR and these three properties, and we present the ERA theorem, asserting that any SMR scheme can only provide at most two of the three properties.

READ FULL TEXT VIEW PDF

page 1

page 2

page 3

page 4

1. Introduction

Managing memory for concurrent data structures is known to be non-trivial. The main problem is that a node that is detached from the data structure and is headed for reclamation, may still be accessed by concurrent threads that gained access to prior to its detachment. Once a detached node is reclaimed, the executing threads are in danger of accessing freed memory, potentially causing a system crash, a segmentation fault, or correctness failure (Michael, 2004b, a). This problem can be prevented using a rigid access discipline with locking, but such locking is not typically used because it is often detrimental to performance, and because locking foils the progress guarantee of non-blocking data structures.

To deal with this problem, SMR schemes were presented. The task of an SMR scheme is to either prevent hazardous accesses, or delay the reclamation of nodes that are still accessible by concurrent threads. With an SMR scheme, nodes are first manually retired, announcing that they are candidates for reclamation (and re-allocation) and the SMR scheme is responsible for determining when a retired node can be safely reclaimed and reused. Retired nodes are typically held in pair-thread retire lists  (Michael, 2004b; Wen et al., 2018; Ramalhete and Correia, 2017; Sheffi et al., 2021a; Singh et al., 2021; Harris, 2001) until they are eligible for reclamation.

Each SMR comes with some benefits over existing schemes, but also with some disadvantages for the concurrent system. In this paper we focus on three good properties that have been mentioned in prior work: Ease of integration, Robustness, and wide Applicability (ERA, in short). We define these properties formally, and present a theorem, asserting that it is not possible for any SMR scheme to deliver all three qualities.

Concurrent data-structure implementations usually adhere to some correctness criteria, typically, linearizability (Herlihy and Wing, 1990), and provide some progress guarantee, typically lock-freedom or wait-freedom (Herlihy and Shavit, 2011; Herlihy, 1991). Interestingly, there is no analogue widely accepted correctness criteria for SMR implementations. Various works have phrased ad-hoc correctness conditions (Kang and Jung, 2020; Michael, 2004b; Balmau et al., 2016; Dice et al., 2016; Meyer and Wolff, 2019b, a). For example, (Michael, 2004b; Ramalhete and Correia, 2017; Wen et al., 2018) introduce the terms protection, or hazardous access, which are specific to the SMR scheme they present, but are not adequate for other schemes. Some previous work prove ad-hoc properties that relate to correctness (Sheffi et al., 2021a; Wen et al., 2018; Singh et al., 2021), but there is no correctness condition that is applicable to all SMR techniques in the literature. Clearly, a reclamation scheme should preserve the original implementation’s correctness (e.g., linearizability) and it is desirable to also keep its progress guarantee. The notion of safety is often used, but with no rigorous acceptable definition available. For example, safety may be interpreted as preventing access to reclaimed memory, but then, optimistic methods (Sheffi et al., 2021a; Cohen and Petrank, 2015a, b), that allow access to reclaimed data, may erroneously be considered as unsafe.

In addition to the lack of an adequate definition for safe memory reclamation, there are also some properties of SMR schemes that have never been rigorously defined. Previous works (Singh et al., 2021; Kang and Jung, 2020; Nikolaev and Ravindran, 2021) listed various desirable SMR properties, which are often used to evaluate the overall quality of an SMR scheme. The most natural properties that we care about include performance (and scalability), progress guarantee, applicability to a wide set of concurrent data-structure implementations, the ease of integration with a given concurrent data structure, and the memory footprint, a.k.a. robustness. While some of these properties are intuitively clear, most of them were not formally defined. Moreover, robustness was only recently introduced, only in the scope of memory reclamation (Balmau et al., 2016; Dice et al., 2016), and is sometimes used to mean different things. Let us briefly discuss the three notions that are at the focus of this paper.

Defining wide applicability is not straightforward. It is not always easy to tell if a given SMR scheme is applicable to a given data structure. For example, it is not trivial to see that the Hazard Pointers (HP) method (Michael, 2004b) and some of its extensions (Ramalhete and Correia, 2017; Wen et al., 2018; Nikolaev and Ravindran, 2020) are not applicable to Harris’s linked-list (Harris, 2001), and it is not clear how to test HP’s applicability to a new data structure. While the applicability notion has been considered previously (Singh et al., 2021; Kang and Jung, 2020; Sheffi et al., 2021a; Cohen and Petrank, 2015a), it was never formally defined.

Robustness has two different meanings in the literature. According to Dice et al. (Dice et al., 2016), an SMR scheme is considered as robust if there is a bound on the number of retired objects that cannot be reclaimed. According to Balmau et al. (Balmau et al., 2016), the number of reclamation-related computational steps should be bounded. While the first definition has subsequently been more widely accepted (Singh et al., 2021; Wen et al., 2018; Sheffi et al., 2021a), both suffer from their informality. First, the bound is not defined. It is unclear whether it should be a constant, or whether it may depend on the size of the data structure, or even the execution length, and in what way. If the data structure requires space, exponential in the number of insert operations, is it robust? If a scheme exhausts the heap even with small data structures, is it robust? Furthermore, what does ”cannot be reclaimed” mean? Various SMR schemes are driven by different reclamation triggers, and a reclamation of a certain set of objects may be postponed, regardless of safety. Moreover, the terminology in existing definitions is not obvious. What is a reclamation-related step? Some SMR schemes change the original implementation’s layout fundamentally, making the difference between the original implementation steps and the newly inserted ones ambiguous. We feel that a formal definition of robustness is missing. The definition should rigorously clarify the concept of memory overhead for SMR schemes.

Finally, we look at how difficult it is to integrate an SMR scheme into a given data structure, i.e., at ease of integration. An SMR scheme typically provides a set of API operations that should be inserted into the given code (e.g., alloc(), retire(), beginOperation() (Harris, 2001; Fraser, 2004; Wen et al., 2018)). Sometimes, the integration also involves changing the original program. E.g., the Automatic Optimistic Access (AOA) scheme (Cohen and Petrank, 2015a) requires the data structure to be in a normalized form (Timnat et al., 2012) for the integration. Free Access (FA) (Cohen, 2018) and Neutralization-Based Reclamation (NBR) (Singh et al., 2021) require that the code is divided into separate read and write phases, and Version-Based Reclamation (VBR) (Sheffi et al., 2021a) provides a designated mechanism for adding code checkpoints. Note that wide applicability and easy integration are independent properties. A scheme can be easily integrated to any given code, but, still, not be applicable to some data-structure implementations.

In this paper, we formally define safe memory reclamation, along with the three desirable properties: robustness, wide applicability and ease of integration. In addition, we present and prove the ERA theorem, asserting that the three properties (Ease of integration, Robustness, and wide Applicability) cannot co-exist. I.e., any SMR scheme can have at most two out of the three desirable properties. This paper is organized as follows: Related work is surveyed in Section 2. In Section 3 we specify the shared-memory model. We formally define safe memory reclamation in Section 4, and the three desirable properties in Section 5. We present the ERA Theorem in Section 6, and conclude in Section 7.

2. Related Work

Detailed surveys of SMR schemes appear in the literature (Sheffi et al., 2021a; Singh et al., 2021; Brown, 2015; Wen et al., 2018). We focus on previous efforts to address and define suitable correctness conditions for SMR schemes and desirable reclamation properties. To the best of our knowledge, while there exist formal methods for verifying safety in manually reclaimed environments (Gotsman et al., 2013; Meyer and Wolff, 2019b, a), and various reclamation schemes have been shown to posses highly desirable properties (e.g., robustness, easy integration and wide applicability), these notions have never been formally defined.

Michael (Michael, 2004b) and Herlihy et al. (Herlihy et al., 2005) were the first to address the need in bounding the amount of memory occupied by removed elements. In both suggested schemes, each thread has a pool of global pointers (called hazard pointers in (Michael, 2004b) or guards in (Herlihy et al., 2005)

), used to postpone the reclamation of certain nodes that might still be in use. Both schemes were designed to provide an upper bound guarantee on the total number of deleted nodes that are not yet eligible for reuse at any given moment. Braginsky et al. 

(Braginsky et al., 2013) suggested a flexible trade-off between space and runtime overhead suggested by these two reclamation schemes. The term Robustness was first introduced by Dice et al. (Dice et al., 2016) and Balmau et al. (Balmau et al., 2016). While the first definition put a limit on the number of deleted objects that could not be reclaimed, the latter bounded the number of steps during any action related to memory reclamation. Subsequently, the first definition (bounding the space overhead) was widely adopted (Ramalhete and Correia, 2017; Wen et al., 2018; Kang and Jung, 2020; Singh et al., 2021; Nikolaev and Ravindran, 2020, 2021; Sheffi et al., 2021a). However, the bound on the space overhead was never specified, and so reclamation schemes with liberal bound (that can potentially exhaust the available memory) (Ramalhete and Correia, 2017; Wen et al., 2018; Nikolaev and Ravindran, 2020) are considered robust. Reference counting-based schemes (Gidenstam et al., 2008; Detlefs et al., 2002; Herlihy et al., 2005) are usually not robust, mainly due to the existence of cyclic structures of retired objects (these cycles can be broken (Sundell, 2005; Correia et al., 2021), at the cost of higher performance overheads).

Although some reclamation schemes are designed for a limited set of data-structures (Braginsky et al., 2013), most methods are designed for a general, wide set of data-structure implementations. A common issue which rises in this context is that many allegedly general schemes (Michael, 2004b; Ramalhete and Correia, 2017; Wen et al., 2018; Nikolaev and Ravindran, 2020; Solomon and Morrison, 2021) require restricted access to deleted nodes. I.e., traversing a deleted node is forbidden. This requirement is problematic, as many state-of-the-art lock-free data-structures allow such traversals (Heller et al., 2005; Harris, 2001; Natarajan and Mittal, 2014; Brown et al., 2014; Ellen et al., 2010) to obtain fast searches. Indeed, this issue was discussed in various works (Cohen and Petrank, 2015a; Fraser, 2004; Kang and Jung, 2020; Brown, 2015). An extensive study by Singh et al. (Singh et al., 2021) lists many popular lock-free data-structures for which such schemes are not applicable. Michael (Michael, 2002) suggested a modification of Harris’s lock-free linked-list (Harris, 2001), which makes it suitable for these reclamation schemes. However, this modification reduces the performance of the linked-list (for more details, see (Cohen and Petrank, 2015a)). Furthermore, it is not a universal construction and it cannot be used to modify other data-structures. Gidenstam et al. (Gidenstam et al., 2008) suggested a more general solution, but their construction adds even higher performance overheads, and involves a complicated manual integration into the given code. Wide applicability was also referred to as generality in (Nikolaev and Ravindran, 2020).

Applicability does not guarantee an easy integration. First, the programmer must issue retire() statements at a location where the node is already detached from the data structure. Also, reclamation schemes often require additional integration. The simplest integration is provided by the seminal EBR scheme (Harris, 2001; Fraser, 2004; Brown, 2015), which solely requires inserting calls to external methods in the beginning and end of each data-structure operation. Other schemes (Michael, 2004b; Wen et al., 2018; Ramalhete and Correia, 2017; Kang and Jung, 2020) provide a slightly more complicated interface, which is still relatively easy for integration. Such interfaces may include an explicit node protection method (preventing the reclamation of retired nodes while they are potentially still in use), and read and write barriers (i.e., code to be executed with any read or write of a data structure field). Such methods can be automatically integrated into an existing implementation, and do not require a significant familiarity with the original code. However, some reclamation schemes (e.g., (Cohen and Petrank, 2015a, b; Brown, 2015; Singh et al., 2021; Sheffi et al., 2021a)) require a more complicated integration procedure. Such schemes may cause the original algorithm to fail in accessing data structure fields and retry. This implies control flow changes to determine where to branch when such a failure occurs. Such integration is non-trivial, and it requires a deep understanding of the underlying data structure. Automatic integration is no longer possible. However, as the ERA Theorem asserts (see Section 6), such harder integration efforts are required to obtain general applicability and a low space overhead.

In addition to performance, robustness, wide applicability, and easy integration, other reclamation properties have been considered in previous work. Singh et al. (Singh et al., 2021), considered consistency, which requires the overall performance to not be affected by workload changes or under a system over-subscription. Nikolaev and Ravindran (Nikolaev and Ravindran, 2021) claimed that a reclamation scheme should be transparent. Meaning, threads can be created and deleted dynamically throughout the execution, and without affecting the scheme’s safety. Cohen and Petrank introduced an optimistic reclamation method (Cohen and Petrank, 2015a, b), which allows threads to read reclaimed memory, while taking care to preserve correctness. Such schemes provide strong robustness and high throughput. Sheffi et al. (Sheffi et al., 2021a) extended optimistic access to writes, providing a fully optimistic solution.

In addition to ensuring desirable properties, avoiding common unwanted side affects is also important. Many reclamation schemes introduce significant memory overheads, as they add extra fields to the data-structure nodes’ layout. E.g., epoch-related data 

(Sheffi et al., 2021a; Wen et al., 2018; Ramalhete and Correia, 2017), or reference counters (Gidenstam et al., 2008; Detlefs et al., 2002; Herlihy et al., 2005). Optimistic methods rely on type-preservation (Cohen and Petrank, 2015a, b; Sheffi et al., 2021a), which means that nodes should be re-allocated to the same type. This is adequate particularly when there is a small number of major data structures in the code. Some reclamation algorithms require special compiler or hardware support (and therefore, are not considered as self-contained (Kang and Jung, 2020)). E.g., DEBRA+ (Brown, 2015) and NBR (Singh et al., 2021) rely on lock-free OS signals to preserve lock-freedom, VBR (Sheffi et al., 2021a, b) and Hyaline (Nikolaev and Ravindran, 2020) rely on a hardware-provided wide CAS instruction, StackTrack (Alistarh et al., 2014) and ThreadScan (Alistarh et al., 2018) rely on transactional memory, Dice et al. (Dice et al., 2016) and PEBR (Kang and Jung, 2020) rely on the existence of process-wide memory fences, and QSense (Balmau et al., 2016) requires control over the OS scheduler.

3. Preliminaries

We follow previous work and use the basic asynchronous shared memory model from (Herlihy, 1991), and related definitions from (Herlihy and Wing, 1990). We consider a fixed set of executing threads, communicating by applying operations on shared objects. An object is an instance of an abstract data type, which specifies a set of possible values, and a set of operations that provide the only means to access it. For example, we define the set data type as a set of integer keys (denoted as set keys), initially empty. Its associated operations are insert(key), delete(key) and contains(key), where key is an integer. The insert(key) operation inserts key into the set and returns true if the set does not already contain key, and returns false otherwise. The delete(key) operation removes key from the set and returns true if the set indeed contains key, and returns false otherwise. The contains(key) operation returns true if the set contains key, and returns false otherwise.

Implementations and Data-Structures

An implementation of an object (or several objects) provides a data-representation by applying primitive memory access operations (e.g., reads, writes, atomic read-modify-write instructions (Herlihy, 1991)) on a set of base objects (i.e., shared memory locations). Specifically, set objects are represented by shared data-structures. Each set key is represented by a node, and each data-structure has a fixed set of entry points (e.g., a linked-list head (Harris, 2001) or a tree root (Natarajan and Mittal, 2014)), which are node pointers. Nodes may contain node pointer fields. We say that a node is a successor of a node (or that is a predecessor of ) if at least one of ’s node pointer fields points to . Accordingly, we say that a node is reachable from a node if there exist nodes such that (1) , (2) , and (3) for every , is a predecessor of . We say that a certain node is reachable if it is reachable from an entry point.

Executions

A step is either a shared-memory access, a local variable access, an operation invocation, or the return from an operation. In all cases, the step includes the executing thread id, the accessed object (when exists), and the respective access input and output values. Steps are considered to be atomic. A configuration specifies the value of each shared memory address and the state of each thread (including the content of its local variables and program counter). The initial configuration is the configuration in which all memory addresses have their initial values and all threads are in their initial states. In particular, all data-structures are initialized, and represent empty sets. An execution is an alternating sequence of configurations and steps, starting from the initial configuration. Specifically, an execution of an implementation is an execution where, starting from the initial configuration, each step is issued according to the given implementation, each memory read matches the preceding configuration, and each memory write is reflected in the following configuration. Given a sub-sequence , we say that is a solo-run if the steps are executed by the same thread.

Histories

An execution is modeled by its history, which is its sub-sequence of operation invocation and response steps. Given an implementation, its set of derived histories is the set of all histories that model executions of that implementation. Given a history and a thread , we denote with the sub-history of , consisting of exactly all the steps executed by in . Similarly, given a history and a shared object (may be a memory word or a data-structure as described above), we denote with the sub-history of , consisting of exactly all the steps executed on in . Accordingly, given a history , a thread and a shared object , we denote with the sub-history of , consisting of exactly all the steps executed by on in . Two histories are equivalent if for every thread , it holds that .

Given a history and an object , we say that is sequential if it begins with an invocation step, and each invocation step (except for possibly the last one) is immediately followed by its matching response. We say that a history is a sequential history if for every object , is sequential. An object is associated with a sequential specification, which is a prefix-closed set of all of its possible sequential histories.

Given a history that contains an operation invocation, we say that this operation is complete in if also contains its matching response. Otherwise, we say that this operation is pending in . A history is complete if all of its contained operations are complete.

Well-Formed Histories

The standard definition of well-formed histories (Herlihy and Wing, 1990) assumes that, given a history and a thread , is a sequence of operation invocations and their immediate matching responses. However, describing the integration of a safe memory reclamation scheme into a given data-structure implementation requires nesting operations. I.e., the reclamation scheme’s operations (e.g., retire(), alloc()) are called in the scope of the data-structure operations. Therefore, we cannot use the standard definition of well-formed histories from (Herlihy and Wing, 1990). Instead, we follow the extended definition from (Attiya et al., 2018). Given a history and an object , we say that is well-formed if for every thread , starts with an invocation step, and is an alternating sequence of invocation steps and their immediate matching response. We say that a history is well-formed if (1) for every object , is well-formed, and (2) for every thread , two of its invocation steps and their respective matching response steps , if precedes and precedes in , then precedes in . A well-formed implementation is an implementation for which all derived histories are well-formed.

Linearizability

A complete history is linearizable if it is well-formed, and for every object , its sequential specification contains a sequential history such that (1) and are equivalent, and (2) if a response step precedes an invocation step in , then it also precedes it in . A history is linearizable if it can be completed (by adding matching response steps to a subset of pending operations in , and removing the rest of ’s pending operations) to a linearizable complete history. A linearizable implementation is an implementation for which all derived histories are linearizable.

Lock-Freedom

We follow Herlihy and Shavit’s definition of lock-freedom (Herlihy and Shavit, 2011). Given an execution and an executing thread , we say that is effective in a configuration if performs the step for some (informally, is not starved by the scheduler). Now, let be an operation invocation by a thread during an execution . We say that is effective if either has a matching response step in , or is effective in for every .

A history provides minimal progress if in every suffix of , some pending effective invocation has a matching response. provides maximal progress if in every suffix of , every pending effective invocation has a matching response. An implementation is lock-free if every respective history provides minimal progress, and some respective history provides maximal progress.

4. Defining Safe Memory Reclamation

In this section we present a formal definition of safe memory reclamation. For ease of presentation, we focus on data-structures that implement set objects. This allows defining a life cycle of a node in the data structure. Extensions to other object types are not difficult.

4.1. Nodes’ Life-Cycles

Following Meyer and Wolff (Meyer and Wolff, 2019b, a) we assume that (for a set implementation) each node goes through stages in a life-cycle, and can be in one of four possible states: unallocated, local, shared, or retired. Initially, a node is unallocated. I.e., its memory is not available for use by the executing threads. After being allocated by a certain thread, the node becomes local. While being local, no thread but the allocating thread has access to this node. In particular, the node cannot be reachable (from an entry point of the data structure), and it cannot represent a set item at this stage. Next, the node may or may not become shared (e.g., by making it reachable). While being shared, the node may become alternately reachable and unreachable, and may also represent a set item. When a node is either local or shared, we also say that it is active. At some point, an executing thread may retire the node, announcing that this node is about to become garbage. Once a thread retires the node, it becomes retired (and cannot be retired again). Note that some nodes never become shared, and therefore become retired after being local. In the lifetime of a node, we assume that a node always becomes unreachable before it becomes retired (generally, nodes can only be reachable while they are shared111This assumption is necessary for most safe memory reclamation schemes. However, there exist scenarios in which it is not needed (Wei et al., 2021).). Finally, a retired node may be reclaimed, meaning that its memory may now be used for re-allocation and its state becomes unallocated again. We consider nodes as logical entities. I.e., after a node returns to being unallocated, a new allocation from the same address is considered as an allocation of a different node (even if both nodes are allocated on the same address).

4.2. Safe and Unsafe Memory Accesses

In this section we define a safe memory access. We think of the memory as segregated into two separate spaces – the system space and the program space. The program space is the area in which the executing threads keep their local and shared nodes and variables. In particular, new allocations are always within the program space, and all nodes reside in the program space, until they are reclaimed. At the time of reclamation, the memory reclamation scheme decides whether to keep the reclaimed nodes for potential subsequent re-allocation in the program space, or to return the node space to the system, in which case, the node moves to system space. If the program attempts to access a node in system space, the result is undefined, and may include a segmentation fault.

From now on, we refer to a given set implementation (with no memory reclamation) as the plain implementation. Note that although plain implementations do not include memory reclamation, they do follow the life cycle defined in Section 4.1. Specifically, they include adequate retire() instructions. Upon integrating a given memory reclamation scheme into a plain implementation, we refer to the derived implementation as the integrated implementation. We further use the plain and integrated terms to describe the respective executions and histories.

Dereferencing a pointer means reading or updating a value in a node whose address is stored in the pointer .222Note that the pointer content is not necessarily equal to the stored address (e.g., marked pointers (Harris, 2001) contain addressees, possibly along with a marked bit). After memory is reclaimed, dereferencing a pointer to it might cause a segmentation fault (Michael, 2004a). Given an execution , a pointer variable , and any configuration in the execution (), let be the last update of in the sub-execution . Namely, for , is updated in , and for every , does not update . The update of in may be an allocation of a new node to or a pointer assignment to from another pointer .333In the following clean formalization, we assume no address arithmetic, and that a pointer points to the head of the object. One can easily extend the definitions below to programs that use pointer arithmetic, as long as there is a clear mapping from pointers to nodes.

Let be the node that is referenced by in . We separate into two cases according to whether is an allocation or an assignment. If the last update of is an allocation in , and the node is not in an unallocated state in any of the intermediate configurations (for ), then we say that is valid in . Otherwise, we say that is invalid in . For example, by definition, is always valid in . The second case is that is a pointer assignment, and let be the pointer whose content is assigned into in . Similarly, we say that is valid in if is valid in , and for all intermediate configurations for , is not in an unallocated state in . If is not valid in , we say that is invalid in . We can now formally define a safe memory accesses.

Definition 4.1 ().

A memory access is unsafe if it dereferences an invalid pointer. It is safe otherwise.

4.3. Defining SMR in the Presence of Unsafe Memory Accesses

When designing a memory reclamation scheme, one must either provide an adequate solution for coping with unsafe memory accesses, or make sure that all memory accesses are safe. Some SMR schemes allow unsafe memory accesses, while taking care to preserve correctness. E.g., AOA (Cohen and Petrank, 2015a) and VBR (Sheffi et al., 2021a) allow reading via invalid pointers, as it is ensured that stale values are always ignored. VBR further allows trying to update the shared memory via invalid pointers, as it is guaranteed that the update fails. Definition 4.2 below encapsulates the conditions for a memory reclamation scheme to be considered as a safe one. Loosely speaking, an SMR should either only use safe memory accesses, or be very careful in how it executes unsafe memory accesses. In particular, it should not access system space (that might end up in a segmentation fault), it should not modify the data on the node (which might have been reclaimed), and it should not use a value read during an unsafe memory access.

Definition 4.2 ().

A memory reclamation scheme is a safe memory reclamation (SMR) scheme with respect to a given plain implementation, if for each respective integrated execution , all memory accesses in all steps are safe, or if all unsafe memory accesses satisfy the following conditions. Let be a step with an unsafe memory access to a memory node via a pointer , then the following three conditions must hold:

  1. In configuration , ’s occupied memory belongs to the program space.

  2. does not update ’s content. Namely, ’s content in equals ’s content in .

  3. If data from the dereferenced node is read into a variable or field (local or shared), then the value in is never used.

The term ”used” in Condition 3 refers to the standard program analysis terminology. In particular, if the modified variable (or field) is read in a step (for some ), then there exists such that overwrites the content of this variable (or field).

Note that we only define safety with respect to a given plain implementation, as most schemes are not necessarily safe when integrated with all existing plain implementations. E.g., the HP (Michael, 2004b) scheme is safe with respect to Michael’s linked-list (Michael, 2002), but is not safe with respect to Harris’s linked-list (Harris, 2001). This implies that HP is not applicable to Harris’s linked-list plain implementation. We further discuss applicability (and HP’s applicability in particular) in Section 5.3 and in Appendix E.

5. Desirable SMR Properties

In this section we present formal definitions for three of the most desirable reclamation scheme properties. Robustness is defined in Section 5.1, easy integration is defined in Section 5.2, and wide applicability is defined in Section and 5.3. One property may affect another in a design. For example, robustness may affect progress, and preserving progress guarantees may affect applicability. But in the definitions, we separate the notions and define each of them independently of the others.

5.1. Robustness (Memory Footprint)

Similarly to (Singh et al., 2021; Sheffi et al., 2021a; Wen et al., 2018), we adopt the definition of robustness from Dice et al. (Dice et al., 2016), which relates to the space overhead or memory footprint of a reclamation scheme. According to this definition, a reclamation scheme is considered robust if a failed or delayed thread cannot totally prevent memory reclamation. This is formalized by a bound on the amount of retired nodes that exist at any point in the execution. As in Section 4.1, retired nodes are nodes that have already been retired, are not in the state of unallocated. This includes nodes that cannot be reclaimed (due to some reclamation condition that the nodes do not satisfy), together with the retired nodes that have simply not yet been reclaimed, typically because some periodic process that reclaims objects has not yet processed them (Singh et al., 2021; Fraser, 2004; Brown, 2015).

To the best of our knowledge, previous work does not specify any general definition for the bound on the number of such objects. It is not clear whether this bound should be a pre-defined constant, may depend on the specific execution, or may depend on the data-structure size. In definitions 5.1-5.2

below we classify the different levels of robustness, according to the bound that an SMR scheme satisfies.

In the following definitions we consider a bound on the number of retired objects that depends on the size of the data structure. The size of the data structure is dynamic and is bounded by the number of active nodes, i.e., the nodes that have been allocated and not yet been retired (see Section 4.1).

We first define the class of robust reclamation schemes. Robustness bounds the number of retired nodes at any time by a function that is asymptotically smaller than the maximum size of the data structure so far in the execution, multiplied by the number of threads. Formally, given an execution , we denote the number of active nodes in by . In addition, we set the function to be .

Definition 5.1 ().

(Robustness) We say that a reclamation scheme is robust if for every integrated execution , there exists a function , such that (1) , and (2) for every configuration , the number of retired nodes in is bounded by .

VBR (Sheffi et al., 2021a) is robust, with being a constant function, bounded by the local retire list size (which does not depend on the execution). This scheme presents the strongest robustness available by an SMR today. Its bound does not depend on the size of the data structure.

HP (Michael, 2004b), AOA (Cohen and Petrank, 2015a), and NBR (Singh et al., 2021) all use hazard pointers (Michael, 2004b) for write protection444The HP scheme uses hazard pointers for read protection as well.. For all three schemes, depends on the pre-defined local retire list size (similarly to VBR) plus the number of hazard pointers. The number of hazard pointers is typically a small constant (e.g., 3 for linked-lists (Michael, 2002; Harris, 2001)), but may also depend on the number of active nodes (e.g., for skip lists with a dynamic number of levels (Aksenov et al., 2020)). While the number of hazard pointers is not guaranteed to be asymptotically smaller than the number of live objects, in all known data structures it is. It is an open question for the study of hazard pointers to bound their number. If the number of hazard pointers is always asymptotically smaller than the number of active nodes, then all three schemes are robust.

Some reclamation schemes do not provide robustness, but a bound still exists. Accordingly, we define a relaxed term of robustness, denoted weak robustness, in Definition 5.2 below. Note that robust schemes are also considered as weakly robust schemes, but not vice-versa.

Definition 5.2 ().

(Weak Robustness) We say that a reclamation scheme is weakly robust if for every integrated execution , there exists a function , such that (1) is polynomial in , and (2) for every configuration , the number of retired nodes in is bounded by .

A weakly robust scheme might incur a larger space overhead. Usually, this happens only at worst-case scenarios, but a large space overhead is theoretically possible. In some hybrids of the epoch-based (Harris, 2001; Fraser, 2004) and pointer-based (Michael, 2004b) reclamation approaches, the execution is divided into epochs, and the number of retired nodes is bounded by the number of active nodes during a certain set of epochs. Given an integrated execution and an epoch , that starts in a configuration , the total number of active nodes during is plus the number of allocations during . In IBR (Wen et al., 2018), each thread might prevent the reclamation of nodes that were active during a small set of reserved epochs555There is a trade-off between IBR’s easy integration and the bound on the number of reserved epochs. For more details, see Section 5.2.. As IBR allows only a constant number of allocations per epoch, the number of retired nodes in a configuration is linear in (which is not asymptotically smaller than ). Therefore, IBR is weakly robust. EBR (Harris, 2001; Fraser, 2004; Brown, 2015) is not even weakly robust. Once a thread is halted, all subsequently allocated nodes can never be reclaimed.

5.2. Easy Integration

We assume that the plain implementation already contains proper retire() invocations. I.e., we do not consider retire() calls installations as part of the reclamation scheme integration. The obvious easiest integrate-able SMR is the EBR scheme. EBR provides a retire() implementation, along with two API operations, beginOp() and endOp(), to be respectively inserted in the beginning and end of every data-structure operation. I.e., any plain implementation can be easily integrated with EBR, and the integration process does not require an understanding of the plain implementation. A definition of easily integrated scheme should obviously include EBR, but other schemes are also not that hard to integrate.

Other examples for easily integrated reclamation methods are the HP (Michael, 2004b) and IBR (Wen et al., 2018) schemes. Both schemes provide designated alloc() and retire() implementations, along with code to replace reading and writing from shared memory. IBR also provides beginOp() and endOp() implementations, to be inserted at the beginning and end of each operation, respectively. Although the integration of both schemes is slightly more complicated than EBR’s, they are still considered as easily integrated, as their integration does not require any familiarity with the original code. Note that the IBR authors provide a weakly robust IBR variant (discussed in Section 5.1), which cannot by easily integrated according to Definition 5.3 below, as this scheme requires inserting roll-back instructions (returning to a previous point in the code) for maintaining a small robustness bound.

Some schemes cannot be easily integrated with many plain implementations. An important example of integration obstacle is the requirement of an SMR to insert roll-back instructions for handling unsafe memory accesses. Namely, when some validation test fails, one needs to return program control to a point in the code from which it is safe to re-execute. Roll-backs are something that we rule out for easy integration. Another undesirable property of an SMR is that it modifies fields of the data structure. An SMR may add fields to a data structure node to be used for its own activity, but expect the SMR to not modify fields that the plain implementation uses. This modularity, encapsulation of information, and separation of concerns between the data structure and the SMR activity are standard principles in software engineering. We stress that marking a pointer to signify a deleted object as in Harris’ linked-list is not a problem, because this is a modification by the data structure to support its own deletion activity, that the SMR is not involved in. In contrast, if the SMR modifies a data structure field, then the programmer that integrates the SMR into the data structure must have an intimate acquaintance with the fields of the data structure to make sure that the SMR does not foil the data structure operations correctness of performance. Such an intimate knowledge, if required during integration, makes the integration difficult.

Some schemes deal with difficult roll-backs by adhering to specific code shapes, which allow easier placement of roll-back mechanisms. AOA (Cohen and Petrank, 2015a) requires that the plain implementation is first transformed into a normalized form (Timnat and Petrank, 2014). NBR (Singh et al., 2021) requires that the code is first divided into separate read and write phases (to be further discussed in Section 5.3), which enable easier rollback call installations. VBR (Sheffi et al., 2021a) relies on linearizability (Herlihy and Wing, 1990) when installing checkpoints and roll-back instructions. We define the easy integration property more rigorously in Definition 5.3 below. This definition excludes reclamation schemes that alter the plain implementation layout and disallows rolling back into wisely chosen code locations. Consequently, it classifies AOA, FA, NBR and VBR as reclamation schemes that cannot be easily integrated. We state the definition and follow up with more explanations.

Definition 5.3 ().

A reclamation scheme is considered an easily integrated scheme if the following conditions hold:

  1. The reclamation scheme is provided as an object.

  2. The reclamation scheme’s API operations may only be inserted in the following code locations: (1) upon the invocation or before the termination of any operation of the plain implementation, (2) as a replacement to alloc() and retire() calls, or (3) as a replacement to primitive memory access operations.

  3. An API operation that replaces a primitive memory access operation, should be a linearizable implementation of that primitive.

  4. The integrated implementation should be well-formed.

  5. The reclamation scheme may add new fields to the node’s layout and it may access these fields, but it cannot access any other node fields.

According to Condition 1, the reclamation scheme should be provided as an object that can be used with all implementations. I.e., as defined in Section 3, it should provide a uniform set of API operations, which are the only way to use its mechanism (for more details, see Section 3). This Condition also ensures that a reclamation scheme is not adjusted in order to fit specific plain implementations. Condition 2 ensures that the integration procedure is indeed relatively easy (as in EBR and IBR). Condition 3 treats shared memory addresses as objects. I.e., each memory address is an object that provides a set of operations (e.g., read, write, read-modify-write), usually implemented via atomic primitives. According to Condition 3, if a reclamation scheme operation replaces such a primitive, then it should implement it in a linearizable manner. By the locality property of linearizability (Herlihy and Wing, 1990), this maintains some level of equivalency between the plain implementation and the integrated one. Condition 4 builds on Condition 1, and treats the plain implementation and the reclamation scheme implementation as implementations of two separate objects. In particular, the meaning of well-formed (as in Section 3) in this sense is that the integration cannot move control from within a reclamation-related operation to a point in the plain implementation. Namely, rollbacks from a reclamation API code back into code of the plain implementation are not allowed. Finally, Condition 5 ensures that the reclamation scheme does not assume anything regarding the node’s layout, and does not access any of its original fields. It may only access new fields that it adds to the node.

AOA, NBR and VBR do not satisfy Definition 5.3, as they do not provide a uniform set of API operations. In particular, inserting roll-back instructions foils Condition 4, as it moves control to an external point in the data-structure code before terminating the current reclamation-related operation execution. Note that the reclamation API should not include an explicit reclamation operation, as reclamation is expected to occur in the scope of the reclamation scheme code.

5.3. Wide Applicability

In this section we define the applicability of a reclamation scheme to a given plain implementation, and define the wide-applicability property accordingly. Our applicability definition includes a proper safety engagement, and a reference to the plain implementation’s correctness and progress guarantees. We use linearizability (Herlihy and Wing, 1990) as our correctness condition, but the definition can be easily adapted to fit other correctness conditions.

Definition 5.4 ().

We say that a reclamation scheme is applicable to a plain implementation if the following hold:

  1. Memory safety: The reclamation scheme is safe with respect to the plain implementation according to Definition 4.2.

  2. Correctness: The integrated implementation is linearizable666Note that linearizability (as defined in Section 3) refers to the object implemented by the plain implementation, regardless of integration. I.e., the integrated implementation is linearizable with respect to this object even when the reclamation scheme cannot be treated as a separate object (see Section 5.2)..

  3. Progress: The integrated implementation provides the same progress guarantee as the plain implementation.

The progress guarantee of a scheme is determined according to (Herlihy and Shavit, 2011), and set to the weakest guarantee of any of its operations. For example, Herlihy and Shavit’s linked-list is lock-free although its contains() operation is wait-free (Herlihy et al., 2020).

Given the definition of applicability of a reclamation scheme to a plain implementation in Definition 5.4, we now define strong applicability and wide applicability of a given SMR. A good property of a reclamation scheme is applicability to as many data structures as possible.

The seminal EBR scheme (Harris, 2001; Fraser, 2004; Brown, 2015) is strongly applicable, as defined in Definition 5.5 below (the full proof appears in Appendix A). It is the strongest scheme in terms of applicability. The only assumption it makes with respect to the plain implementation is that retire() instructions are properly installed (for more details, see Section 4.1).

Definition 5.5 ().

(Strong Applicability) We say that a reclamation scheme is strongly applicable if it is applicable to every plain implementation.

While strong applicability is highly desirable, to the best of our knowledge, EBR is the only scheme that satisfies it. There are still different extents of applicability for the various reclamation schemes in the literature. We are interested in reclamation schemes that, while not applicable to any imaginable data structure, are still widely applicable to many known data structures. To accurately define widely applicable reclamation schemes, we adopt the definition from previous work (Singh et al., 2021), that defines a large class of well-known and widely-used concurrent data-structure implementations (e.g., (Harris, 2001; Natarajan and Mittal, 2014; Heller et al., 2005; Ellen et al., 2010; Howley and Jones, 2012)), all applicable to the NBR reclamation scheme. This class is described in (Singh et al., 2021) as containing all data-structure implementations that can be divided into separate interleaving read and write phases. We provide the formal definition of this class of data-structure implementations in Appendix C. We denote such implementations as access-aware data-structure implementations, and define wide applicability accordingly:

Definition 5.6 ().

(Wide Applicability) We say that a reclamation scheme is widely applicable if it is applicable to all access-aware data-structure implementations.

Not all SMRs are widely applicable. We show in Appendix E that HP, IBR and HE are not widely applicable.

6. The ERA Theorem

In Section 5 we showed that there exists a reclamation scheme which is both widely applicable and easily integrated (EBR). Assuming that the number of hazard pointers (Michael, 2004b) is asymptotically smaller than the data-structure size (see Section 5.1), there also exists a reclamation scheme which is both robust and widely applicable (NBR), and a reclamation scheme which is both robust and easily integrated (HP). In this section we prove the main theorem of this paper:

Theorem 6.1 ().

Any memory reclamation scheme can provide at most two of the following three guarantees: robustness, easy integration and wide applicability.

In fact, we prove a stronger result, namely that even weak robustness (see Definition 5.2) cannot be achieved when easy integration and wide applicability are provided. This stronger result immediately implies Theorem 6.1.

In Appendix D we show that Harris’ linked-list is access-aware, and therefore, a widely applicable reclamation scheme must be also applicable to Harris’s linked-list implementation (Harris, 2001). For completeness, Harris’s plain implementation, including retire() calls, is presented in the supplementary material. The main idea in the proof is to assume in a way of contradiction that an SMR does satisfy all three properties. In this case, it must be applicable to Harris’ linked-list, and we then build a specific execution that is not safe for this concurrent linked-list implementation. Thus, no SMR can satisfy all three properties.

As described in Section 3, the list API provides the insert(), delete() and contains() operations, and the nodes comprise of two fields – an immutable key and a next pointer to the node’s successor in the list. The list maintains two sentinel nodes, head and tail, with the respective and keys, that are never removed from the list. Nodes are logically inserted into the list by physically linking them into the list and making them reachable, and are logically deleted from the list by marking their next pointer (for more details, see (Harris, 2001; Herlihy et al., 2020)). Note that after a node is marked for deletion, it is not necessarily unlinked by the thread that had previously marked it, as it might be unlinked during a concurrent operation. However, the marked node is guaranteed to be unlinked and retired before the delete() operation returns.

All three API operations use the search() auxiliary method, which is in charge of (1) locating a given key in the list, and (2) unlinking logically deleted (i.e., marked) nodes from the list. This method traverses the list (by following next pointers) until it finds the first unmarked node with a key greater than or equal to the searched key. After locating such a node, the method might try to physically unlink a sequence of marked nodes from the list. The crucial point here is that marked nodes are not unlinked during the traversal. As opposed to Michael’s implementation (Michael, 2002) (that was originally designated to fit HP (Michael, 2004b)), when the search() method encounters a marked node, it just continues its traversal.

We prove Theorem 6.1 by constructing a specific execution. Let be some fixed constant, and assume threads are executing Harris’s linked list plain implementation, integrated with a widely applicable memory reclamation scheme. By Definition 5.6, the given reclamation scheme is applicable to this plain implementation. In particular, by Definition 5.4, the integrated implementation must provide the same progress guarantee as Harris’s algorithm, namely lock-freedom. Now, assume by contradiction that the scheme is both weakly robust and easily integrated.

Initially (stage a in Figure 1), there are two reachable nodes in the list (besides the and sentinels). Assume that starts executing a delete(3) operation. It calls search(3), and starts its traversal by reading head’s next pointer, which is currently referencing node 1. At this stage, the scheduler moves control to , which executes a delete(1) operation. marks node 1 for deletion (stage b) and physically unlinks it from the list (stage c). Next, executes an insert(3) operation (stage d) and a delete(2) operation (stages e-f). In a similar way, continues calling an alternating sequence of insert(n+1) and delete(n) (starting from ).

Figure 1. Lower bound illustration ().

For every , let be the integer such that is the configuration after returns from the delete(n) execution. Note that for every , after executes insert(n+1), there are four active nodes in the system (, , and ), as the rest of the nodes are already retired before their respective delete() operations return. Finally, after executes delete(n), there are three active nodes in the system (, and ), and the nodes are already retired. Therefore, for every , . As the integrated reclamation scheme is weakly robust, there exists a function , which is polynomial in , such that the number of retired nodes in is bounded by .

Let ( must exist as the number of threads and are constants). In , for every , node has already been retired, and as , by Definition 5.2, at least one of the nodes must already be reclaimed. In , if a node , , is already retired but not yet reclaimed, then it must be marked and pointing to node via its pointer. To see that the latter is true, recall that, as the scheme is easily integrated, according to Condition 5 from Definition 5.3, the reclamation scheme does not update pointers.

Starting from , let the scheduler apply a solo-run by . I.e., is the only effective thread in and on (for more details, see Section 3). continues its traversal from node 1. As all nodes along its path are marked, the traversal’s stopping condition does not hold for all nodes along its path (which have keys smaller than , and that have not been reclaimed yet), and should continue its traversal. By Condition 4 from Definition 5.3 (forcing well-formedness), must return from this read operation (as implemented by the reclamation scheme) before it continues its execution. As is the only effective thread, and as lock-freedom is guaranteed (see Section 3), every such read operation by indeed terminates. In addition, according to Condition 3 from Definition 5.3, as long as a node (for any ) is not reclaimed, a read of its next pointer by must return a marked reference to the memory (either previously or currently) occupied by node . Therefore, as long as does not encounter a reclaimed node, every read of a pointer must terminate, returning a (marked) reference to the next node.

Recall that there exists an already reclaimed node (for some ), such that for every , node ’s next pointer is marked, and is pointing to node . Eventually, must dereference a pointer stored in the memory formerly occupied by . It must assign the content of an invalid pointer (as has already been reclaimed) to a local pointer variable, , performing an unsafe memory access by Definition 4.1. By Definition 5.4, the given reclamation scheme is safe with respect to this plain implementation, and therefore, by Definition 4.2, ’s local pointer must be overridden before dereferences it. By Condition 2 and 3 from Definition 5.3, does not perform any updates before dereferencing , as it contains a marked reference – a contradiction to the scheme’s applicability to the given plain implementation. Therefore, a memory reclamation scheme cannot provide robustness, easy integration and wide applicability.

Discussion.

We proved this result using a specific data structure (Harris’s linked-list). This is enough to prove the impossibility in Theorem 6.1, but it actually provides a stronger result. Even if one tries to achieve robustness and ease of integration only for Harris’s linked-list, then this attempt must fail. Therefore, other weaker interpretations of applicability that only require applicability to smaller sets of implementations must still respect the impossibility of Theorem 6.1, as long as the notion of applicability for an SMR requires it to be applicable to Harris’s list (among other implementations).

An interesting open question is to characterize implementations that are similar to Harris’ linked-list. Namely, that memory reclamation with (weak) robustness and ease of integration cannot be obtained for them. It is interesting to understand which data structures require special care.

Finally, we stress the practical importance of this theorem. In order to apply HP to Harris’s linked-list, Maged (Michael, 2002) modified Harris’s implementation to disallow the simultaneous removal of multiple consecutive nodes from the list. While this allowed applying HP to the list, Michael’s implementation was slower than the original implementation, see for example the evaluation in (Cohen and Petrank, 2015a). Thus, avoiding implementations that are non-trivial for memory reclamation may reduce performance of the concurrent data structures.

7. Conclusion

In this paper we proposed some theoretical foundation for safe memory reclamation for concurrent data structures. We provided definitions for safe memory reclamation and for three fundamental desirable properties: robustness, easy integration and wide applicability. We then proved that no reclamation scheme can provide all three desirable properties. I.e., robust reclamation schemes (with limited space overhead) should either be designed for specific implementations, or come with a relatively complicated manual for proper integration. Open questions include the formalization of additional interesting properties of SMRs and formal safety proofs for existing SMR schemes.

References

  • V. Aksenov, D. Alistarh, A. Drozdova, and A. Mohtashami (2020) The splay-list: a distribution-adaptive concurrent skip-list. arXiv preprint arXiv:2008.01009. Cited by: §5.1.
  • D. Alistarh, P. Eugster, M. Herlihy, A. Matveev, and N. Shavit (2014) Stacktrack: an automated transactional approach to concurrent memory reclamation. In Proceedings of the Ninth European Conference on Computer Systems, pp. 1–14. Cited by: §2.
  • D. Alistarh, W. Leiserson, A. Matveev, and N. Shavit (2018) Threadscan: automatic and scalable memory reclamation. ACM Transactions on Parallel Computing (TOPC) 4 (4), pp. 1–18. Cited by: §2.
  • H. Attiya, O. Ben-Baruch, and D. Hendler (2018) Nesting-safe recoverable linearizability: modular constructions for non-volatile memory. In Proceedings of the 2018 ACM Symposium on Principles of Distributed Computing, pp. 7–16. Cited by: §3.
  • O. Balmau, R. Guerraoui, M. Herlihy, and I. Zablotchi (2016) Fast and robust memory reclamation for concurrent data structures. In Proceedings of the 28th ACM Symposium on Parallelism in Algorithms and Architectures, pp. 349–359. Cited by: §1, §1, §1, §2, §2.
  • A. Braginsky, A. Kogan, and E. Petrank (2013) Drop the anchor: lightweight memory management for non-blocking data structures. In Proceedings of the twenty-fifth annual ACM symposium on Parallelism in algorithms and architectures, pp. 33–42. Cited by: §2, §2.
  • T. A. Brown (2015) Reclaiming memory for lock-free data structures: there has to be a better way. In Proceedings of the 2015 ACM Symposium on Principles of Distributed Computing, pp. 261–270. Cited by: Appendix A, §2, §2, §2, §2, §5.1, §5.1, §5.3.
  • T. Brown, F. Ellen, and E. Ruppert (2014) A general technique for non-blocking trees. In Proceedings of the 19th ACM SIGPLAN symposium on Principles and practice of parallel programming, pp. 329–342. Cited by: §2.
  • N. Cohen and E. Petrank (2015a) Automatic memory reclamation for lock-free data structures. ACM SIGPLAN Notices 50 (10), pp. 260–279. Cited by: §1, §1, §1, §2, §2, §2, §2, §4.3, §5.1, §5.2, §6.
  • N. Cohen and E. Petrank (2015b) Efficient memory management for lock-free data structures with optimistic access. In Proceedings of the 27th ACM symposium on Parallelism in Algorithms and Architectures, pp. 254–263. Cited by: §1, §2, §2, §2.
  • N. Cohen (2018) Every data structure deserves lock-free memory reclamation. Proc. ACM Program. Lang. 2 (OOPSLA), pp. 143:1–143:24. External Links: Link, Document Cited by: §1.
  • A. Correia, P. Ramalhete, and P. Felber (2021) OrcGC: automatic lock-free memory reclamation. In Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 205–218. Cited by: §2.
  • D. L. Detlefs, P. A. Martin, M. Moir, and G. L. Steele Jr (2002) Lock-free reference counting. Distributed Computing 15 (4), pp. 255–271. Cited by: §2, §2.
  • D. Dice, M. Herlihy, and A. Kogan (2016) Fast non-intrusive memory reclamation for highly-concurrent data structures. In Proceedings of the 2016 ACM SIGPLAN International Symposium on Memory Management, pp. 36–45. Cited by: §1, §1, §1, §2, §2, §5.1.
  • F. Ellen, P. Fatourou, E. Ruppert, and F. van Breugel (2010) Non-blocking binary search trees. In Proceedings of the 29th ACM SIGACT-SIGOPS symposium on Principles of distributed computing, pp. 131–140. Cited by: §2, §5.3.
  • K. Fraser (2004) Practical lock-freedom. Technical report University of Cambridge, Computer Laboratory. Cited by: Appendix A, Appendix C, §1, §2, §2, §5.1, §5.1, §5.3.
  • A. Gidenstam, M. Papatriantafilou, H. Sundell, and P. Tsigas (2008) Efficient and reliable lock-free memory reclamation based on reference counting. IEEE Transactions on Parallel and Distributed Systems 20 (8), pp. 1173–1187. Cited by: §2, §2, §2.
  • A. Gotsman, N. Rinetzky, and H. Yang (2013) Verifying concurrent memory reclamation algorithms with grace. In European Symposium on Programming, pp. 249–269. Cited by: §2.
  • T. L. Harris (2001) A pragmatic implementation of non-blocking linked-lists. In International Symposium on Distributed Computing, pp. 300–314. Cited by: Appendix A, Appendix B, Appendix C, Appendix D, Appendix D, §1, §1, §1, §2, §2, §3, §4.3, §5.1, §5.1, §5.3, §5.3, §6, §6, footnote 2.
  • S. Heller, M. Herlihy, V. Luchangco, M. Moir, W. N. Scherer, and N. Shavit (2005) A lazy concurrent list-based set algorithm. In International Conference On Principles Of Distributed Systems, pp. 3–16. Cited by: §2, §5.3.
  • M. Herlihy, V. Luchangco, P. Martin, and M. Moir (2005) Nonblocking memory management support for dynamic-sized data structures. ACM Transactions on Computer Systems (TOCS) 23 (2), pp. 146–196. Cited by: §2, §2.
  • M. P. Herlihy and J. M. Wing (1990) Linearizability: a correctness condition for concurrent objects. ACM Transactions on Programming Languages and Systems (TOPLAS) 12 (3), pp. 463–492. Cited by: §1, §3, §3, §5.2, §5.2, §5.3.
  • M. Herlihy, N. Shavit, V. Luchangco, and M. Spear (2020) The art of multiprocessor programming. Newnes. Cited by: Appendix B, §5.3, §6.
  • M. Herlihy and N. Shavit (2011) On the nature of progress. In International Conference On Principles Of Distributed Systems, pp. 313–328. Cited by: §1, §3, §5.3.
  • M. Herlihy (1991) Wait-free synchronization. ACM Transactions on Programming Languages and Systems (TOPLAS) 13 (1), pp. 124–149. Cited by: §1, §3, §3.
  • S. V. Howley and J. Jones (2012) A non-blocking internal binary search tree. In Proceedings of the twenty-fourth annual ACM symposium on Parallelism in algorithms and architectures, pp. 161–171. Cited by: §5.3.
  • J. Kang and J. Jung (2020) A marriage of pointer-and epoch-based reclamation. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 314–328. Cited by: Appendix E, §1, §1, §1, §2, §2, §2, §2.
  • R. Meyer and S. Wolff (2019a) Decoupling lock-free data structures from memory reclamation for static analysis. Proceedings of the ACM on Programming Languages 3 (POPL), pp. 1–31. Cited by: §1, §2, §4.1.
  • R. Meyer and S. Wolff (2019b) Pointer life cycle types for lock-free data structures with memory reclamation. Proceedings of the ACM on Programming Languages 4 (POPL), pp. 1–36. Cited by: §1, §2, §4.1.
  • M. M. Michael (2002) High performance dynamic lock-free hash tables and list-based sets. In Proceedings of the fourteenth annual ACM symposium on Parallel algorithms and architectures, pp. 73–82. Cited by: §2, §4.3, §5.1, §6, §6.
  • M. M. Michael (2004a) ABA prevention using single-word instructions. IBM Research Division, RC23089 (W0401-136), Tech. Rep. Cited by: §1, §4.2.
  • M. M. Michael (2004b) Hazard pointers: safe memory reclamation for lock-free objects. IEEE Transactions on Parallel and Distributed Systems 15 (6), pp. 491–504. Cited by: Figure 2, Appendix E, §1, §1, §1, §1, §2, §2, §2, §4.3, §5.1, §5.1, §5.2, §6, §6.
  • A. Natarajan and N. Mittal (2014) Fast concurrent lock-free binary search trees. In Proceedings of the 19th ACM SIGPLAN symposium on Principles and practice of parallel programming, pp. 317–328. Cited by: §2, §3, §5.3.
  • R. Nikolaev and B. Ravindran (2020) Universal wait-free memory reclamation. In Proceedings of the 25th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 130–143. Cited by: §1, §2, §2, §2.
  • R. Nikolaev and B. Ravindran (2021) Snapshot-free, transparent, and robust memory reclamation for lock-free data structures. In Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pp. 987–1002. Cited by: §1, §2, §2.
  • P. Ramalhete and A. Correia (2017) Brief announcement: hazard eras-non-blocking memory reclamation. In Proceedings of the 29th ACM Symposium on Parallelism in Algorithms and Architectures, pp. 367–369. Cited by: Figure 2, Appendix E, §1, §1, §1, §2, §2, §2, §2.
  • G. Sheffi, M. Herlihy, and E. Petrank (2021a) VBR: version based reclamation. In 35th International Symposium on Distributed Computing, DISC 2021, October 4-8, 2021, Freiburg, Germany (Virtual Conference), S. Gilbert (Ed.), LIPIcs, Vol. 209, pp. 35:1–35:18. External Links: Link, Document Cited by: Appendix C, §1, §1, §1, §1, §1, §2, §2, §2, §2, §2, §4.3, §5.1, §5.1, §5.2.
  • G. Sheffi, M. Herlihy, and E. Petrank (2021b) Vbr: version based reclamation. In Proceedings of the 33rd ACM Symposium on Parallelism in Algorithms and Architectures, pp. 443–445. Cited by: §2.
  • A. Singh, T. Brown, and A. Mashtizadeh (2021) NBR: neutralization based reclamation. In Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 175–190. Cited by: Appendix C, Appendix D, Appendix E, §1, §1, §1, §1, §1, §1, §2, §2, §2, §2, §2, §2, §5.1, §5.1, §5.2, §5.3.
  • D. Solomon and A. Morrison (2021) Efficiently reclaiming memory in concurrent search data structures while bounding wasted memory. In Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 191–204. Cited by: §2.
  • H. Sundell (2005) Wait-free reference counting and memory management. In 19th IEEE International Parallel and Distributed Processing Symposium, pp. 10–pp. Cited by: §2.
  • S. Timnat, A. Braginsky, A. Kogan, and E. Petrank (2012) Wait-free linked-lists. In International Conference On Principles Of Distributed Systems, pp. 330–344. Cited by: §1.
  • S. Timnat and E. Petrank (2014) A practical wait-free simulation for lock-free data structures. ACM SIGPLAN Notices 49 (8), pp. 357–368. Cited by: §5.2.
  • Y. Wei, N. Ben-David, G. E. Blelloch, P. Fatourou, E. Ruppert, and Y. Sun (2021) Constant-time snapshots with applications to concurrent data structures. In Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 31–46. Cited by: footnote 1.
  • H. Wen, J. Izraelevitz, W. Cai, H. A. Beadle, and M. L. Scott (2018) Interval-based memory reclamation. ACM SIGPLAN Notices 53 (1), pp. 1–13. Cited by: Figure 2, Appendix E, §1, §1, §1, §1, §1, §2, §2, §2, §2, §2, §5.1, §5.1, §5.2.

Appendix A EBR is Strongly Applicable

In this section we prove that the seminal EBR reclamation scheme (Fraser, 2004; Harris, 2001; Brown, 2015) is strongly applicable. According to Definition 5.4 and 5.5, this means that, given any plain implementation: (1) EBR is a safe memory reclamation with respect to that implementation, (2) integrating EBR maintains linearizability, and (3) integrating EBR maintains the same progress guarantee, provided by the plain implementation.

In EBR, the execution is divided into epochs. The executing threads maintain a shared epoch counter and a shared announcements array. Upon each invocation of a data-structure operation, the begin_op() operation is called. During the begin_op() operation, the executing thread reads the current global epoch and announces it in the shared announcements array. Then, it checks whether all other threads have announced the current epoch (or a quiescent state). If they have, it increments the shared epoch counter. Before a data-structure operation returns, the end_op() operation is called, for announcing a quiescent state in the global announcements array.

In addition to the global epoch counter and announcements array, each thread maintains three local retire lists, containing the nodes retired during the last three epochs, respectively (EBR’s retire() operation only inserts the retired node into the current epoch’s retire list). Once the global epoch counter shows epoch , the retire list for epoch can be reclaimed.

Now, assume a plain data-structure implementation. We are going to show that EBR is applicable to the given implementation. According to Definition 5.4, we need to show that (1) EBR is safe with respect to the given implementation, (2) the integrated implementation is linearizable, and (3) the integrated implementation provide the plain implementation’s progress guarantee.

First, we are going to prove that every memory access, during any integrated execution, is safe by Definition 4.1. Recall that according to Section 4, the plain implementation already contains retire() calls. It is guaranteed that each node is retired at most once, and that nodes are no longer reachable after being retired. Let be an integrated execution, and assume by contradiction that is an unsafe memory access by a thread . W.l.o.g., assume that is the first unsafe memory access during . By Definition 4.1, dereferences an invalid pointer, , at . Let be the last assignment into before , and let be the node referenced by at . As is invalid at , and by the choice of , was reclaimed at some point between and .

Let be ’s retire epoch. The global epoch at must be at least (as is already reclaimed). This means that must have announced an epoch which is at least , upon the invocation of the current operation. As started its operation during an epoch which is at least , has not been reachable (and obviously, not local to ) at any point during the operation execution. By our assumptions in Section 3, cannot have access to ’s previously occupied memory address. In particular, its local pointer, , could not have referenced this address – a contradiction. Therefore, every memory access, during any integrated execution, is safe by Definition 4.1. By Definition 4.2, EBR is safe with respect to every plain implementation.

Given that EBR is safe with respect to any given plain implementation, the history of every EBR-integrated execution is equivalent to some history of an execution of the plain implementation. By the definition of linearizability, every EBR-integrated implementation maintains the linearizability of the respective plain implementation. Finally, EBR maintains any progress guarantee, as its three operations all consist of a finite number of steps. As EBR is applicable to every plain implementation by Definition 5.4, it is strongly applicable by Definition 5.5.

1:private Window *search(key)
2:     retry: while (true) do      
3:          Node *pred = head
4:          Node *pred_next = head next
5:          Node *curr = pred_next
6:          Node *curr_next = curr next
7:          while (isMarked(curr_next) ——           
8: key ¡ curr key) do
9:               if (!isMarked(curr_next)) 
10:                    pred = curr
11:                    pred_next = curr_next                
12:               curr = getRef(curr_next)
13:               if (curr == tail)  break                
14:               curr_next = curr next           
15:          if (pred_next == curr) 
16:               if (isMarked(curr next)) 
17:                    goto retry
18:               else return new Window(curr, pred)                          
19:          if (CAS(&pred next, pred_next, curr)) 
20:               Node *tmp = pred_next
21:               Node *tmp_next = tmp next
22:               if (isMarked(curr next))  goto retry
23:               else return new Window(curr, pred)                               
24:
25:public boolean contains(key)
26:     Window *window = search(key)
27:     Node *curr = window curr
28:     return !isMarked(curr next) curr key == key
29:
30:public boolean insert(key)
31:     Node *new_node = alloc(key)
32:     while (true) do
33:          Window *window = search(key)
34:          Node *pred = window pred
35:          Node *curr = window curr
36:          if (curr key == key) 
37:               retire(new_node)
38:               return false           
39:          new_node next = curr
40:          if (CAS(&pred next, curr, new_node)) 
41:               return true                
42:
43:public boolean delete(key)
44:     while (true) do
45:          Window *window = search(key)
46:          Node *pred = window pred
47:          Node *curr = window curr
48:          if (curr key key) 
49:               return false           
50:          Node *succ = getRef(curr next)
51:          Node *marked = getMarked(succ)
52:          if (!CAS(&curr next, succ, marked)) 
53:               continue           
54:          if (!CAS(&pred next, curr, succ)) 
55:                search(key)           
56:          retire(curr)
57:          return true      
Algorithm 1 Based on Harris’s Non-Blocking Linked-List Implementation

Appendix B Harris’s Linked-List

Harris’s plain implementation, including retire() calls (see lines 37 and  56), is presented in Algorithm 1. As described in Section 3, the list API provides the contains (lines 25-28), insert (lines 30-41) and delete (lines 43-57) operations, and the nodes comprise of two fields – an immutable key and a next pointer to the node’s successor in the list.

The list maintains two sentinel nodes, head and tail, with the respective and keys, that are never removed from the list. Nodes are logically inserted into the list by physically linking them into the list and making them reachable (see line 40). Nodes are logically deleted from the list by marking their next pointer (for more details, see (Harris, 2001; Herlihy et al., 2020)). Note that after a node is marked for deletion in line 52, it is not necessarily unlinked by the thread that had previously marked it, as it might be unlinked during a concurrent operation. However, the marked node is guaranteed to be unlinked and retired (in line 56) before the operation returns in line 57.

Besides the provided API operations, the implementation uses the search() auxiliary method (lines 1-23), which is in charge of (1) locating a given key in the list (the respective node and its predecessor are returned in line 18 or 23), and (2) unlinking logically deleted nodes from the list (in line 19). Marked nodes are not necessarily unlinked during the traversal. The search() method tries to unlink a series of marked nodes (line 19) only if at least one of the output candidates is marked. In such case, it is guaranteed that the condition in line 15 does not hold.

Appendix C Access-Aware Data-Structure Implementations

In this section we formalize the definition of access-aware data-structure implementations. This definition relates to the class of implementations, originally defined in (Singh et al., 2021).

As briefly discussed in Section 5.3, such implementations can be divided into separate read-only and write phases (the latter type may also include reads). Besides these two phase types, each write phase is preceded by a conceptual reservations phase. The reservation phase should be added during the reclamation scheme integration into the code, and is intended for protecting potential hazardous accesses during the write phase. Since the reservations phase is not reflected in the given plain implementation (and since some reclamation schemes do not employ protection at all (Sheffi et al., 2021a; Fraser, 2004; Harris, 2001)), we chose to avoid it in our definition. The separation into phases itself is enough for inserting reservation commands before write phases.

The original definition roughly states two conditions for classifying proper read-only and write phases:

  1. During a read-only phase, shared nodes can be read only if pointers to them were obtained during the current phase.

  2. During a write phase, shared nodes can be accessed (either for reads or for writes) only if they were reserved prior to the current phase.

For formalizing the above two conditions, the notion of obtaining pointers in condition 1 should be properly phrased, and an alternative condition for the reservation in condition 2 should also be provided.

For the latter condition, it suffices to demand that all shared memory accesses during a write phase, do not dereference newly updated pointers. As long as all dereferenced pointers (either for reads or for writes) are already obtained during the last read-only phase, any integrated reclamation scheme can add the desired reservations phase between the two phases.

The notion of obtaining a pointer can be treated in a similar way to our treatment of safe memory access in Section 4.2. Given an execution , a pointer variable , and any configuration in the execution (), let be the last update of in the sub-execution . Namely, for , is updated in , and for every , does not update . The update of in may be an allocation of a new node to , a pointer assignment from a global variable (according to Section 3, it must be a data-structure entry point), or a dereference of a pointer (e.g., when reading the next pointer of a data-structure node).

Given , we say that is -permitted at , if one of the following holds:

  1. is an allocation, and at , the respective allocated node is still local to the allocating thread.

  2. is a pointer assignment from a global variable, and .

  3. is a dereference of a pointer , , and is -permitted at .

We are now going to define the class of access-aware data-structure implementations, as follows: a plain implementation is considered as access-aware, if it can be divided into separate alternating read-only and write phases, such that for every derived execution and a step , it holds that:

  1. If is a shared-memory read into a local pointer, , during a read-only segment that started in (for some ), then is -permitted at .

  2. If is a shared-memory read into a local pointer, , during a write segment that started in (for some ), then is either an allocation of a new node, a read of a global variable, or a dereference of another pointer variable, , that was -permitted at (given that the last read-only segment began at ).

  3. If is a shared-memory write, dereferencing a pointer , then the current phase is a write phase. and was -permitted when the last read-only phase ended (given that the last read-only segment began at ).

Note that retirements of nodes are not considered as shared memory reads or writes. This definition does not relate to retirement at all.

Appendix D Harris’s Linked-List is an Access-Aware implementation

In this section we are going to prove that Harris’s linked-list implementation (Harris, 2001) (as presented in Appendix B) is an access-aware data-structure implementation. This has already been shown in (Singh et al., 2021). However, for the completeness of our main theorem proof (see Section 6), we also show it for our definition (from Appendix C).

We are first going to divide the code into separate read-only and write phases, as follows:

Searches

The first read-only phase starts in line 2. If the method returns in line 18, then a new write phase begins. If the method does not return in line 18, just before line 19 is executed, a new write phase begins, and if the condition in line 22 holds, the write phase ends and a new read-only phase begins. Note that in any case, the search method always returns during a write phase. This means that both and can be assumed to be reserved at this point.

Contains

After returning from the search call in line 26, lines 27-28 are executed during a write phase.

Inserts

The operation starts accessing shared memory when the search auxiliary method is called in line 33. After it returns, the rest of the loop iteration is executed in a write phase.

Deletes

The operation starts accessing shared memory when the search auxiliary method is called in line 45. After it returns, the rest of the loop iteration is executed in a write phase, unless the search method is called again in line 55. Then, a new read-only phase is initiated, an so on.

It remains to show that the above separation into phases indeed adheres to the terms defined in Appendix C. As every read-only phase starts with reading the global variable in line 3 and dereferencing permitted pointers (can be proven by induction), the read-only terms always hold. In addition, the and pointers, accessed during the executions of all three operations, are permitted in the last respective read-only-phase. As the new node, allocated in line 31, is not shared before the execution of line 40, accessing it is also permitted. Finally, as mentioned in Appendix C, the retirement in line 56 is also permitted. Therefore, Harris’s linked-list implementation (Harris, 2001) (as presented in Appendix B) is an access-aware data-structure implementation.

Appendix E Incompatibility to Harris’s Linked-List

Many popular reclamation schemes (e.g., HP (Michael, 2004b), HE (Ramalhete and Correia, 2017), and IBR (Wen et al., 2018)) are known (Singh et al., 2021; Kang and Jung, 2020) to not be applicable to the plain implementation, presented in Algorithm 1 (and by the proof from Appendix D, are not widely applicable). Specifically, HP, IBR, and HE, provide safety via pointers protection. I.e., dereferencing is executed in the following manner: the thread first reads the pointer’s content, then publishes some data (this data differs between the schemes), and then reads its content again. If the content has not changed between the reads, it is guaranteed that the referenced node is protected, and the thread may safely access it. Relying on pointers protection is problematic, since a stable pointer value (i.e., the same value is read twice) does not necessarily guarantee safe access to the referenced node.

Figure 2. An example for HP’s (Michael, 2004b), HE’s (Ramalhete and Correia, 2017), and IBR’s (Wen et al., 2018) limited applicability, using the linked-list implementation from Algorithm 1.

Consider the scenario depicted in Figure 2 (the scenario depicted in Figure 1 can also serve as an example). Initially (stage a), the list contains two nodes (15 and 76). At this stage, invokes insert(58), calls the search() auxiliary method in line 33, starts traversing the list, obtains a local pointer to node 15, protects it, and is then halted by the scheduler. During stage b, some other thread successfully inserts node 43 into the list777Inserting node 43 after node 15 is protected by is crucial for deriving a contradiction to HE’s and IBR’s safety. For HP, it can also work if the protection is executed after node 43 is inserted.. During stage c, invokes delete(43) and invokes delete(15). After both threads find the respective removal window in line 45, both of them successfully mark their victim node in line 52. At this point, invokes delete(44). During its traversal, it physically unlinks nodes 15 and 43 (by setting ’s reference to point to node 76). Nodes 15 and 43 are eventually retired by and (respectively), before returning from their delete() executions (line 56). cannot reclaim node 15, as it is protected by , but successfully reclaims node 43, as it is not protected by any thread. Eventually, returns in line 49, as there does not exist any node with a 44 key. After terminates, continues its execution, reads node 15’s pointer, protects node 43’s address and re-reads node 15’s pointer. After making sure node 15’s pointer is stable, dereferences its pointer to node 43’ address (pre-reclamation), which is already reclaimed. This scenario obviously foils safety, as node 43’s memory is either returned to the OS or contains another node at this stage. Note that even if the memory, previously occupied by node 43, is not returned to the OS, this access foils safety. HP, HE and IBR do not employ a mechanism for dealing with stale values, as depicted in Definition 4.2.