Efficient, Near Complete and Often Sound Hybrid Dynamic Data Race Prediction (extended version)

04/15/2020
by   Martin Sulzmann, et al.
0

Dynamic data race prediction aims to identify races based on a single program run represented by a trace. The challenge is to remain efficient while being as sound and as complete as possible. Efficient means a linear run-time as otherwise the method unlikely scales for real-world programs. We introduce an efficient, near complete and often sound dynamic data race prediction method that combines the lockset method with several improvements made in the area of happens-before methods. By near complete we mean that the method is complete in theory but for efficiency reasons the implementation applies some optimizations that may result in incompleteness. The method can be shown to be sound for two threads but is unsound in general. We provide extensive experimental data that shows that our method works well in practice.

READ FULL TEXT VIEW PDF

page 1

page 2

page 3

page 4

08/01/2018

What Happens - After the First Race? Enhancing the Predictive Power of Happens - Before Based Dynamic Race Detection

Dynamic race detection is the problem of determining if an observed prog...
09/07/2019

Predicting All Data Race Pairs for a Specific Schedule (extended version)

We consider the problem of data race prediction where the program's beha...
12/08/2021

Efficient Data Race Detection of Async-Finish Programs Using Vector Clocks

Existing data race detectors for task-based programs incur significant r...
10/30/2020

Optimal Prediction of Synchronization-Preserving Races

Concurrent programs are notoriously hard to write correctly, as scheduli...
07/19/2019

Online Set-Based Dynamic Analysis for Sound Predictive Race Detection

Predictive data race detectors find data races that exist in executions ...
04/30/2019

Dependence-Aware, Unbounded Sound Predictive Race Detection

Data races are a real problem for parallel software, yet hard to detect....
01/25/2019

Fast, Sound and Effectively Complete Dynamic Race Detection

Writing concurrent programs is highly error-prone due to the nondetermin...

1. Introduction

We consider verification methods in the context of concurrently executing programs that make use of multiple threads, shared reads and writes, and acquire/release operations to protect critical sections. Specifically, we are interested in data races. A data race arises if two unprotected, conflicting read/write operations from different threads happen at the same time.

Detection of data races via traditional run-time testing methods where we simply run the program and observe its behavior can be tricky. Due to the highly non-deterministic behavior of concurrent programs, a data race may only arise under a specific schedule. Even if we are able to force the program to follow a specific schedule, the two conflicting events many not not happen at the same time. Static verification methods, e.g. model checking, are able to explore the entire state space of different execution runs and their schedules. The issue is that static methods often do not scale for larger programs. To make them scale, the program’s behavior typically needs to be approximated which then results in less precise analysis results.

The most popular verification method to detect data races combines idea from run-time testing and static verification. Like in case of run-time testing, a specific program run is considered. The operations that took place are represented as a program trace. A trace reflects the interleaved execution of the program run and forms the basis for further analysis. The challenge is to predict if two conflicting operations may happen at the same time even if these operations may not necessarily appear in the trace right next to each other. This approach is commonly referred to as dynamic data race prediction.

The challenge of a dynamic data race prediction algorithm is to be efficient, sound and complete

. By efficient we mean a run-time that is linear in terms of the size of the trace. Sound means that races reported by the algorithm can be observed via some appropriate reordering of the trace. If unsound, we refer to wrongly a classified race as a

false positive. Complete means that all valid reorderings that exhibit some race can be predicted by the algorithm. If incomplete, we refer to any not reported race as a false negative.

Our interest is to study various efficient dynamic data race prediction algorithms and consider their properties when it comes to soundness and completeness. There are two popular methods to obtain an efficient algorithm: Happens-before (Lamport, 1978) and lockset (Dinning and Schonberg, 1991). We review both methods and state-of-the art algorithms that rely on these methods in the upcoming Section 3. Our idea is to combine happens-before and lockset in a novel way. This leads to a new hybrid dynamic data race prediction algorithm. We provide extensive experimental results covering performance as well as precision.

In this work, we make the following contributions:

  • We propose a novel efficient dynamic race prediction method that combines the lockset method with ideas found in the happen-before based SHB (Mathur et al., 2018) and WCP (Kini et al., 2017) algorithms. The method is shown to be complete in general and sound for the case of two threads (Section 4).

  • We give a detailed description of how to implement our proposed method (Section 5). We provide for an algorithm that overall has quadratic run-time. This algorithm can be turned into a linear run-time algorithm by sacrificing completeness. For practical as well as contrived examples, incompleteness is rarely an issue.

  • We carry out extensive experiments covering a large set of real-world programs as well as a collection of the many challenging examples that can be found in the literature. For experimentation, we have implemented our algorithm as well as its contenders in a common framework. We measure the performance, time and space behavior, as well as the precision, e.g. ratio of false positives/negatives etc. Measurements show that our algorithm performs well compared to state-of-the art algorithms such as ThreadSanitizer, FastTrack, SHB and WCP (Section 6).

Section 3 covers earlier efficient dynamic data race prediction algorithms. Section 7 summarizes further related work. Section 8 concludes. The appendix contains optional material such as proofs, extended examples, optimization details etc.

2. Preliminaries

We introduce some notations and we formally define the dynamic data race prediction problem. The development largely follows similar recent works, e.g. consider Kini et al. (2017); Mathur et al. (2018).

Run-Time Events and Traces

We assume concurrent programs making use of shared variables and acquire/release (a.k.a. lock/unlock) primitives. Further constructs such as fork and join are omitted for brevity. We assume that programs are executed under the sequential consistency memory model (Adve and Gharachorloo, 1996). This is a standard assumption made by most data race prediction algorithms. The upcoming condition (CR1) in Definition 2.5 reflects this assumption.

Programs are instrumented to derive a trace of events when running the program. A trace is of the following form.

Definition 2.1 (Run-Time Traces and Events).

Besides , we sometimes use symbols and to refer to events.

A trace is a list of events. We use the notation a list of objects is a shorthand for . We write to denote the concatenation operator among lists. For each event , we record the thread id number in which the event took place, written . We write and to denote a read and write event on shared variable at position . We write and to denote a lock and unlock event on mutex . The number represents the position of the event in the trace. We sometimes omit the thread id and the position for brevity.

We often use a tabular notation for traces where we introduce for each thread a separate column and the trace position can be identified via the row number. Below, we find a trace specified as list of events (on the right) and its corresponding tabular notation (on the left).

We introduce some helper functions. For trace , we assume some functions to access the thread id and position of . We define if for some traces . We define , , and to extract the trace position from an event. We assume that the trace position is correct: If then for some events and trace . We often drop the component and write and for short.

Given a trace , we can also access an event at a certain position . We define if where .

For trace , we define to be the set of events in . We write if .

For trace , we define the projection of onto thread where (1) for each where we have that , and (2) for each where we have that . That is, the projection onto a thread comprised of all events in that thread and the program order remains the same.

Besides accurate trace positions, we demand that acquire and release events are in a proper acquire/release order.

Definition 2.2 (Proper Acquire/Release Order).

We say a trace satisfies a proper acquire/release order if the following conditions (AR1-3) are satisfied.

Condition (AR1): For there exists where . No other acquire/release event on occurs in between trace positions and .

Condition (AR2): For each , if where then there exists where .

We refer to each pair that satisfies the above conditions as a pair of matching acquire-release events.

Condition (AR3): For each two matching-acquire release pairs and
where we have that .

Conditions (AR1-2) ensure that the lock semantics is respected. Condition (AR2) covers the case that an acquire is without matching release. This happens for traces that result from programs that terminated within a critical section. Condition (AR3) states that critical sections for two distinct lock variables and cannot overlap.

We say a trace is well-formed if trace positions in are correct and satisfies a proper acquire/release order.

Trace Reordering and Data Race

We define the set of predictable pairs of conflicting events that are in a data race. Conflicting events are combinations of write-write, write-read and read-write pairs that involve the same variable. By predictable we mean that the data race can be exposed by reordering the trace such that the two the conflicting events appear right next to each other in the trace.

To define reorderings concisely, we introduce some helpful definitions for read/write events and critical sections.

Definition 2.3 (Read/Write Events).

Let be a trace. We define as the set of all read/write events in on some variable . We define as the union of for all variables .

Let be a subset of events in . Then, we define .

Let where either both are write events or one of them is a read and the other is a write event. We assume that and result from different threads. Then, we say that and are two conflicting events.

Let where is a read event and is a write event. We say that is the last write for w.r.t.  if (1) appears before in the trace, and (2) there is no other write event on in between and in the trace.

Definition 2.4 (Critical Section).

Let be a trace.
We write to denote a critical section in if the following conditions (CS1-2) are satisfied.

Condition (CS1): is a subtrace of .

Condition (CS2): The pair is a matching pair of acquire-release events.

We write if is one of the events in the critical section.

We often write as a short-form for a critical section .

We write to denote that the critical section is part of the trace .

We write to refer to and to refer to .

If the thread id does not matter, we write for short and so on. If the lock variable does not matter, we write for short and so on.

Definition 2.5 (Correct Reordering).

Let be a well-formed trace. Let be a trace such that (CR1) for each thread id  we have that is a subtrace of , (CR2) for each read event in where is the last write for w.r.t. , we have that is in and is also the last write for w.r.t. , and (CR3) satisfies a proper acquire/release order. Then, we say that is a correctly reordered prefix of . In such a situation, we write .

We only reorder existing events and the program order for each thread remains the same (see (CR1)). Each read observes the same last write (see (CR2)) and the order of acquire/release events is proper (see (CR3)). Hence, trace is a prefix of a permutation of trace where results from choosing a different sequence of interleaved execution steps that leaves the program order, last write property and lock semantics intact. Trace positions in may no longer be accurate because of the reordering events. For convenience, we keep trace positions as defined by to uniquely identify events when comparing elements from and .

Critical sections represent atomic units and the events within cannot be reordered. However, critical sections themselves may be reordered. Each reordering of the original traces reflects a certain schedule that represents a possible interleaved execution of the program. We distinguish between schedules that leave the order of critical sections unchanged (trace-specific schedule), and schedules that reorder critical sections (alternative schedule).

Definition 2.6 (Schedule).

Let be a well-formed trace and some correctly reordered prefix of .

We say represents the trace-specific schedule in if the relative position of (common) critical sections (for the same lock variable) in and is the same. For lock variable and critical sections where appears before in we have that and appears before in . Otherwise, we say that represents some alternative schedule.

Example 2.7 ().

Consider the well-formed trace

Then, is a correctly reordered prefix of where represents an alternative schedule.

For each correctly reordered prefix (schedule), we identify conflicting events that are in a data race. A data race is represented as a pair of events where and are in conflict and we find a schedule where appears right before in the trace. We refer to as a predictable data race pair because the race is predicted by a reordered trace.

The condition that appears right before is useful to clearly distinguish between write-read and read-write races. We generally assume that for each read there is an initial write. Write-read race pairs are linked to write-read dependencies where a write immediately precedes a read. Read-write race pairs indicate situations where a read might interfere with some other write, not the read’s last write. For write-write race pairs it turns out if appears right before for some reordered trace then can also appear right before by using a slightly different reordering. Hence, write-write pairs and are equivalent and we only report the representative where appears before in the original trace.

Below are the formal definitions for predictable data race pairs followed by some example.

Definition 2.8 (Initial Writes).

We say a trace satisfies the initial write property if for each read event on variable in there exists a write event on variable in where .

The initial write of a read does not necessarily need to occur within the same thread. It is sufficient that the write occurs before the read in the trace. From now on we assume that all traces satisfy the initial write assumption, as well as the well-formed property.

Definition 2.9 (Predictable Data Race Pairs).

Let be a trace. Let be a correctly reordered prefix of . Let . We refer to as a predictable data race pair if (a) are two conflicting events in , and (b) appears right before in the trace .

We say is a write-read race pair if is a write and is a read. We say is a read-write race pair if is a read and is a write. We say is a write-write race pair if both events are writes.

We write for predictable write-read, read-write and write-write race pairs and traces and as specified above. For write-write pairs we demand that .

We define . We refer to as the set of all predictable data pairs derivable from .

We define
. We refer to as the set of all trace-specific predictable data race pairs derivable from .

Our characterization of predictable data races does not rule out deadlocks. A predictable data race may not be feasible because if we would try to follow the schedule that is meant to exhibit the race we run into a deadlock. This is a known issue, see (Kini et al., 2017). Checking for deadlocks and ruling out their presence is beyond the scope of this paper.

Example 2.10 ().

Consider the following trace where we use the tabular notation.

For each event we consider the possible candidates for which forms a predictable race pair. We start with event .

For we immediately find (1) . We also find (2) by putting in between and . There are no further combinations where can appear right before some . For instance, is not valid because otherwise the ‘last write‘ condition (CR2) in Definition 2.5 is violated.

Consider . We find (3) because is a correctly reordered prefix of . It is crucial that we only consider prefixes. Any extension of that involves would violate the ‘last write‘ condition (CR2) in Definition 2.5. For there is another pair (4) . The pair is not a valid write-read race pair because and result from the same thread and therefore are not in conflict.

Consider . We find pairs (5) and (6) . For instance (5) is due to the prefix . The remaining race pairs are (7) and (8) .

Pairs (1) and (3) as well as pairs (2) and (8) are equivalent write-write race pairs. When collecting all predictable race pairs we only keep the representatives (1) and (2). Hence, we find where each race pair is represented by the numbering schemed introduced above. There are no critical sections and therefore no alternative schedules. Hence, .

Example 2.11 ().

Consider the following trace (on the left) and the set of predictable and trace-specific race pairs (on the right).

There are no read-write races in this case. The pair results from the correctly reordered prefix (alternative schedule) The pair is not in because represents some alternative schedule and there is no trace-specific schedule where the write and read appear right next to each other.

We summarize. For each race pair there is a reordering where appears right before in the reordered trace. Each write-write race pair is also a write-write race pair . We choose the representative where appears before in the original trace. For each write-read race pair we have that is ’s last write. Each read-write race pair represents a situation where the read can interfere with some other write .

Next, we review dynamic data race prediction algorithms that attempt to identify all predictable write-write, write-read and read-write data race pairs.

Definition 2.12 ().

Let be a trace and A some algorithm that reports pairs of conflicting events.

We say A is efficient if the time to report pairs is linear in the size of the trace.

We say A is sound if each pair reported is a predictable data race in .

We say A is complete if all predictable data races in are reported.

If unsound, we refer to wrongly a classified data race pair as a false positive. If incomplete, we refer to any not reported predictable data race pair as a false negative.

3. Efficient Race Prediction Methods

We review earlier works on efficient dynamic data race prediction that rely on happens-before and lockset methods.

3.1. Happens-Before Methods

A popular method to obtain a data race prediction algorithm is to derive from the trace a happens-before relation among events. If for two conflicting events, neither event happens before the other event, a trace reordering exists under which both events can appear next to each other. However, depending on the happens-before relation, the trace reordering to exhibit the race may not be correct. A happens-before based algorithm may therefore be unsound. A happens-before based algorithm may also be incomplete if two conflicting events that are in a race are ordered such that one happens before the other. Next, we review the main works in this area.

First, we review the classic happens-before (HB) relation introduced by Lamport (1978). HB-based algorithms are neither sound nor complete. Then, we consider some recent works that attempt to make HB either more sound, or more complete. We also cover race prediction algorithms that implement these ordering relations.

3.1.1. Lamport’s Happens-Before

Here is Lamport’s happens-before relation (Lamport, 1978).

Definition 3.1 (Happens-Before (HB) (Lamport, 1978)).

Let be a trace. We define a relation among trace events as the smallest strict partial order such that the following conditions holds:

Program order (PO)::

Let where and . Then, .

Release-acquire dependency (RAD)::

Let such that (1) , and (2) for all where , and we find that is not an acquire event on . Then, .

We refer to as the happens-before (HB) relation.

We often write Lamport’s happens-before relation as HB relation for short. The HB relation has been implemented by a number of dynamic race prediction algorithms, e.g. see Flanagan and Freund (2010); Pozniansky and Schuster (2003). The Djit algorithm by Pozniansky and Schuster (Pozniansky and Schuster, 2003)

makes use of vector clocks 

(Fidge, 1992; Mattern, 1989) to establish the HB relation. The FastTrack algorithm by Flanagan and Freund (2010)

employs a more optimized representation of vector clocks that uses the thread’s time stamp, referred to as an epoch. Details of vector clocks and epochs follow later.

Djit and FastTrack are efficient and run in linear time. However, the HB method and algorithms that implement HB are neither complete nor sound as the following examples.

Example 3.2 ().

We illustrate incompleteness and unsoundness via the the following two traces.

First, we consider the trace on the right. We apply Definition 3.1 for the construction of the HB relation. Hence, we find that (1) , , (2) , , (3) . Relations (1+2) result from the program order condition. Relation (3) results from the release-acquire dependency. Via transitivity we conclude that . The two writes on are ordered and therefore no race is reported.

However, there is a correctly reordered prefix under which events and are in a race. Consider where represents an alternative schedule. Hence, we find that the HB method is incomplete.

Next, we consider the trace on the left. We find that and . Hence, the conflicting events and are unordered.

However, the pair is not a predictable data race because there is no correct reordering as we otherwise would violate condition (CR2) in Definition 2.5. Condition (CR3) is important because the value read at trace position may affect the control flow of the program. Hence, the earlier write on must remain in the same (relative) position w.r.t. the subsequent read. Hence, the HB method is unsound.

The above shows that incompleteness of the HB relation results from the fact that a trace-specific order among critical section is enforced. See condition (RAD) in Definition 3.1. Unsoundness results from the fact that the HB relation ignores write-read dependencies. Next, we consider some recent works that tackle the soundness and incompleteness issue.

3.1.2. Schedulable Happens-Before

Mathur, Kini and Viswanathan (Mathur et al., 2018) extend the HB relation by including write-read dependencies.

Definition 3.3 (Schedulable Happens-Before (SHB) (Mathur et al., 2018)).

Let be a trace. We define a relation among trace events as the smallest partial order such that and the following condition holds:

Write-read dependency (WRD)::

Let such that and for all where and we find that is not a write event on . Then, .

We refer to as the schedulable happens-before relation.

Mathur, Kini and Viswanathan provide for an efficient algorithm, referred to as SHB, that implements the schedulable happens-before relation. We will also abbreviate the schedulable happens-before relation as SHB and write SHB algorithm and SHB relation to distinguish between the two.

Mathur and coworkers show that only the first race reported by FastTrack is predictable but all subsequent races reported may be false positives. Their SHB algorithm comes with the guarantee that all races reported are predictable. Recall Example 3.2. We additionally find and therefore the events and are ordered and not in a race.

Like the HB relation, the SHB relation orders critical sections based on the order manifested in the trace. Recall Example 3.2. Under the SHB relation we find that . Hence, the SHB relation as well as the algorithm are incomplete in general.

However, the SHB relation is complete for all trace-specific predictable data race pairs where is the set of all such pairs. Recall Definition 2.9.

Definition 3.4 (SHB WRD Race Pairs).

Let be a trace. Let be two conflicting events such that is a write and a read where and there is no such that . Then, we say that is a SHB WRD race pair.

The SHB WRD race pair definition characterizes all trace-specific write-read races. We can state that trace-specific schedule race pairs are either SHB WRD races or events and are concurrent w.r.t. the SHB relation.

Proposition 3.5 (SHB Trace-Specific Soundness and Completeness).

Let be a trace. Let be two conflicting events. Then, iff either (1) is a write-write or read-write pair and neither nor , or (2) is a SHB WRD race pair.

Sulzmann and Stadtmüller (2019) show that the SHB algorithm does not report all trace-specific predictable data races. They introduce a refinement of the SHB algorithm that is able to collect all trace-specific predictable data races. This improved prediction capability comes at some additional cost. Unlike, the SHB algorithm that has a linear run-time, the algorithm by Sulzmann and Stadtmüller (2019) has a quadratic run-time.

3.1.3. Weak-Causally Precedes

Relations HB and SHB enforce a strict order among critical sections based on the order found in the trace. See the release-acquire dependency (RAD) condition in Definition 3.1. Hence, both relations are unable to predict races that result from alternative schedules.

Kini, Mathur and Viswanathan (Kini et al., 2017) introduce a weaker form of happens-before order among acquire/release events, referred to weak-causally precedes (WCP). Based on the WCP relation we are able to predict races that result from alternative schedules. Importantly, the WCP relation still has an efficient implementation as shown by Kini et al. (2017). The WCP relation is defined as follows.

Definition 3.6 (Release Events).

Let be a trace. We define as the set of all release events in T on some variable .

Definition 3.7 (Weak-Causally Precedes (WCP) (Kini et al., 2017)).

Let be a trace. We define a relation among trace events as the smallest partial order that satisfies condition PO as well as the following conditions:

WCP Critical Sections::

Let be two conflicting events. Let , be two critical sections where , , . Then, .

WCP-Ordered Critical Sections::

Let , be two critical sections. Let be two release events where and . Let be two events where , and . Then, .

HB Closure::

is closed under left and right composition with .

We refer to as the weak-causally precedes (WCP) relation.

The WCP Critical Sections Condition is weaker compared to the RAD condition. Recall Example 3.2. Unlike HB and SHB, WCP does not enforce a strict order among the two critical sections. Hence, the two writes on are unordered under WCP. Hence, the WCP relation is able to predict races that result from alternative schedules.

WCP is also, like SHB, complete for all trace-specific data race pairs.

Proposition 3.8 (WCP Trace-Specific Completeness).

Let be a trace. Let be two conflicting events such that . Then, we have that neither nor .

However, WCP is still incomplete in general as shown by the following example.

Example 3.9 ().

Consider . Events and are in a predictable data race as witness by the following correctly reordered prefix

Based on the WCP Critical Sections Condition we find that . In combination with the HB Closure Condition we find that based on the following reasoning

Hence, under WCP we cannot predict the above predictable data race.

Like FastTrack, the WCP algorithm that implements the WCP relation is shown to be sound for the first race predicted (Kini et al., 2017). Subsequent races reported may be false positives.

One of the reasons for unsoundness is that write-read dependencies are ignored (like in case of the HB relation). Recall the earlier Example 3.2. Events and are unordered under the WCP relation.

3.2. Lockset Method

A different method is based on the idea to compute the set of locks that are held when processing a read/write event (Dinning and Schonberg, 1991). We refer to this set as the lockset. If two conflicting events share the same lock then both events must belong to two distinct critical sections involving lock . As critical sections are mutually exclusive, two conflicting events that share the same lock cannot be in a data race.

Below, we define the lockset.

Definition 3.10 (Lockset).

Let be a trace For each read/write event we define . We refer to as the lockset of .

The lockset is easy to compute and leads to an efficient data race prediction algorithm. For two conflicting events we simply check if . If the intersection of the locksets of and is non-empty, then cannot be a predictable data race because and are protected by the same lock. Otherwise, is a potential data race pair.

This shows that the lockset method is complete. The issue is that an empty intersection is not a sufficient criteria for a data race. Hence, the lockset method is unsound. Recall Example 3.2.

To make lockset more sound, hybrid methods include some of happens-before order to rule out conflicting events that are clear false positives. For example, the ThreadSanitizer (TSan) algorithm by Serebryany and Iskhodzhanov (2009) only applies the lockset comparison for events that are not ordered under the program order (see Definition 3.1).

3.3. Discussion

HB SHB WCP Lockset
sound
complete
semi-complete
alternatives
Table 1. HB, SHB, WCP and Lockset Summary

Table 1 summarizes the properties of the HB, SHB and WCP ordering relations as well as the lockset method. By semi-complete we refer to the property that for a specific schedule (e.g. trace-specific) all predictable races can be detected. By alternatives we refer to the ability to predict races that result from distinct schedules.

SHB and WCP are semi-complete. See Propositions 3.5 and 3.8. HB is weaker compared to SHB. Hence, HB is semi-complete as well. HB and WCP are unsound in general and therefore algorithms that rely on these relations are prone to false positives. The same applies to Lockset. All happens-before relations are incomplete which means that we may miss races (false negatives). Lockset on the other hand is complete and therefore also semi-complete.

Could we make any of the relations SHB and WCP more sound and more complete? We believe this is difficult by just using happens-before relations.

4. SHB and WCP meet Lockset

Our idea is to further refine the lockset method by incorporating ideas introduced by the SHB and WCP relation. We adopt the WRD condition from SHB but do not impose the RAD condition because RAD enforces a strict order among critical sections. Instead, we adapt the WCP Critical Sections condition.

Definition 4.1 (WRD + Weak WCP).

Let be a trace. We define a relation among trace events as the smallest partial order that satisfies conditions PO and WRD as well as the following condition:

Weak WCP::

Let be two events. Let , be two critical sections where , and . Then, .

We refer to as the WRD + Weak WCP (W3) relation.

Compared to the WCP relation, the W3 relation additionally imposes the WRD condition. On the other hand, for W3 we no longer impose the WCP-Ordered Critical Sections and HB Closure conditions. Instead, W3 imposes the Weak WCP condition. The essential difference compared to WCP is that W3 only orders critical sections in case of write-read dependency conflicts whereas WCP orders critical sections in case of any conflict such as write-write, read-write etc. Recall Example 3.9 where due to the WCP Critical Sections condition we have that . The W3 relation does not impose any order among the critical sections for this example.

To summarize. The W3 relation is made weaker compared to WCP to avoid incompleteness. The W3 relation is made stronger to avoid unsoundness due to write-read dependencies. On its own, the W3 relation is still too weak and therefore we pair up the W3 relation with the lockset check. Based on this combination we are able to identify all predictable data race pairs. We still may face false positives. Hence, we refer to conflicting events identified by the Lockset-W3 method as potential race pairs.

We first cover potential write-write and read-write pairs of conflicting events.

Definition 4.2 (Lockset + W3 Write-Write and Read-Write Check).

Let be a trace where are two conflicting events such that (1) , (2) neither nor , and (3) is a write-write or read-write race pair. Then, we say that is a potential Lockset-W3 data race pair.

To cover write-read pairs of conflicting events we adapt the WRD race pair definition for SHB to the W3 setting.

Definition 4.3 (Lockset + W3 WRD Check).

Let be a trace. Let be two conflicting events such that is a write and a read where , and there is no such that . Then, we say that is a potential Lockset-W3 WRD data race pair.

Definition 4.4 (Potential Race Pairs via Lockset + W3).

We write to denote the set of all potential Lockset-W3 (and WRD) data race pairs as characterized by Definitions 4.2 and 4.3.

Unlike the SHB setting where all race pairs are predictable, the Lockset-W3 method only identifies potential pairs because not every pair in is predictable. For examples we refer to Appendix C. However, covers all predictable data race pairs.

Proposition 4.5 (Lockset + W3 Completeness).

Let be a trace. Let such that . Then, we find that .

The result follows from the fact that relation does not rule out any of the correct reorderings and schedules that are covered in Definition 2.5.

We can also state the Lockset-W3 check is sound under certain conditions.

Proposition 4.6 (Lockset + W3 Soundness for Two Threads).

Let be a trace that consists of at most two threads. Then, any potential Lockset-W3 data race pair is also a predictable data race pair.

In comparison, the WCP relation is neither sound nor complete for the case of two threads. See Examples 3.2 and 3.9.

Like the HB and WCP relation, Lockset-W3 is unsound in general. Our experiments show that the Lockset-W3 method works well in practice. The number of false positives is small compared to the number of data races reported.

5. Implementation

We provide for an algorithm that implements the Lockset-W3 method to compute the set . Our algorithm combines ideas found in FastTrack (Flanagan and Freund, 2010), SHB (Mathur et al., 2018) and WCP (Kini et al., 2017). For example, we employ vector clocks and the more optimized epoch representation (FastTrack) and a history of critical sections (WCP) to compute the W3 relation. We track write-read dependencies (SHB) and immediately report write-read races.

Like the above algorithms, our algorithm also processes events in a stream-based fashion. Unlike the above algorithms, write-write and read-write races are not immediately reported while processing events. We follow the SHB algorithm (Sulzmann and Stadtmüller, 2019) and report all such potential races in some post-processing phase. Before diving into the technical details of our algorithm, we motivate the need for post-processing via a simple example.

Example 5.1 ().

Consider the trace . There are two (actual) data race pairs: and . Single-pass algorithms will miss the pair as for efficiency reasons only the most recent concurrent events are kept. At the time, we encounter the conflicting events and , event has been ‘replaced’ by .

To catch such cases we need to maintain a history of replaced events that could be part of a potential data race pair. In a first pass, the SHB algorithm (Sulzmann and Stadtmüller, 2019) employs a variant of the SHB algorithm to maintain the history of replaced events by connecting replaced events via edges (E). Pairs of concurrent events represented by epochs (E) are accumulated. Hence, the name SHB. In some post-processing pass, SHB traverses edges using the so far accumulated concurrent pairs as a starting point. Thus, all trace-specific data race pairs can be detected. We adapt this idea to our setting. By limiting the history, post-processing can be integrated into the first pass. This might lead to incompleteness but yields an efficient, linear run-time algorithm.

5.1. W3po Algorithm

1:procedure acquire() 2:      3:      4:      5:      6:end procedure 1:procedure release() 2:      3:      4:      5:      6:end procedure 1:function w3() 2:     for  do 3:         for  do 4:              if  then 5:                   6:              end if 7:         end for 8:     end forreturn V 9:end function
1:procedure write()
2:     
3:     
4:     
5:     
6:     
7:     
8:     
9:     
10:     
11:end procedure
1:procedure read()
2:     
3:     if  then
4:         
5:     end if
6:     
7:     
8:     
9:     
10:     
11:     
12:     
13:end procedure
Algorithm 1 W3PO algorithm (first pass)

We first consider the multi-pass algorithm, referred to as W3PO. The first pass of W3PO is specified by Algorithm 1. Events are processed in a stream-based fashion. For each event we find a procedure that deals with this event.

We compute the lockset for read/write events and check if read/write events are concurrent by establishing the W3 happens-before relation. To check if events are in W3 relation we make use of vector clocks and epochs. We first define vector clocks and epochs and introduce various state variables maintained by the algorithm that rely on these concepts.

For each thread  we compute the current set of locks held by this thread. We use to avoid confusion with the earlier introduced set that represents the lockset for event . We have that where is the set at the time we process event . Initially, for all threads .

The algorithm also maintains several vector clocks.

Definition 5.2 (Vector Clocks).

A vector clock is a list of time stamps of the following form.

We assume vector clocks are of a fixed size . Time stamps are natural numbers and each time stamp position corresponds to the thread with identifier .

We define to synchronize two vector clocks by building the point-wise maximum.

We write to access the time stamp at position . We write as a short-hand for incrementing the vector clock at position by one.

We define vector clock to be smaller than vector clock , written , if (1) for each thread , ’s time stamp in is smaller or equal compared to ’s time stamp in , and (2) there exists a thread where ’s time stamp in is strictly smaller compared to ’s time stamp in .

If the vector clock assigned to event is smaller compared to the vector clock assigned to , then we can argue that happens before . For we find that and .

For each thread we maintain a vector clock . For each shared variable we find vector clock to maintain the last write access on . Initially, for each vector clock