Security is a primary concern for software systems. Programming errors like out-of-bounds memory accesses and inexhaustive input validation are responsible for dangerous and costly incidents. Accordingly, many mechanisms exist for protecting systems against common vulnerabilities like memory-safety errors and input injection. Among the most effective automated approaches is coverage-based greybox fuzzing [10, 9] popularized by the American Fuzzy Lop (afl) fuzzer , which has uncovered copious critical vulnerabilities in the core software libraries underlying a vast swath of software systems. Its efficacy is largely due to broad applicability and direct feedback: afl employs a genetic input-generation algorithm using only lightweight instrumentation-based monitoring to determine concrete vulnerability-witnessing inputs. Importantly, it works without user interaction and even source code, and sidesteps the computational and methodological bottlenecks imposed by traditional program analysis techniques.
Fuzzers like afl-fuzz  target traditional temporal safety properties , i.e., properties concerning individual system executions. Important security aspects like secure information flow  are two-safety properties , i.e., properties concerning pairs of executions, and cannot be precisely characterized as traditional safety properties . Secure information flow is particularly insidious given the potential for side channels, through which adversaries may infer privileged information about the data accessed by a program, e.g., by correlating execution-time differences with control-flow differences. Side-channels are difficult for programmers to reason about since they are generally hardware-dependent. Furthermore, while compiler optimizations are obliged to respect traditional safety properties, they are not designed to respect two-safety properties, and can thus generate insecure machine code which otherwise would be secure without compiler optimizations.
In this work we extend fuzzing to two-safety properties. In particular, we present ct-fuzz,111ct-fuzz: the fuzzer for constant time. https://github.com/michael-emmi/ct-fuzz which enables the precise discovery of timing-based information leaks while retaining the efficacy of afl-fuzz . Via self-composition , we reduce testing two-safety properties to testing traditional safety properties by program transformation, effectively enabling the application of any fuzzer. Our implementation tackles three basic challenges. First, the program under test must efficiently simulate execution-pairs of the original; to avoid overhead, each execution’s address space is copy-on-write shared. Second, structured input-pairs must be derived from random fuzzer-provided input; leaks are only witnessed when inputs differ solely by secret content. Third, leakage-inducing actions must be monitored; leaks are witnessed when, e.g., control flow or memory-access traces diverge, given inputs differing solely by secret content.
Our implementation focuses on timing leaks related to program control-flow and cpu cache; the name ct-fuzz refers to the constant-time property, asserting that the control-flow and accessed memory locations of two executions differing solely by secrets are identical. Besides constant-time, we also apply ct-fuzz to finer-grained cache models to validate secure yet non-constant-time programs which ensure identical cache timing despite potentially-divergent memory-access patterns, e.g., by preloading all accessed memory into the cache. While our implementation focuses on side-channel leakage related to timing, it is extensible to other forms of leakage, e.g., power or electro-magnetic radiation, by further extending the instrumentation mechanism to record other aspects of program behavior. In principle, ct-fuzz could be extended to expose violations to any two-safety property expressible as equality between two program traces.
In the remainder we describe several aspects of the ct-fuzz tool. Sections 2 and 3 cover foundations and implementation concerns in testing two-safety properties. Sections 4 and 5 cover ct-fuzz’s software architecture and basic functionality. Sections 6 and 7 describe case studies and evaluation, demonstrating the effective application of ct-fuzz to many cryptographic libraries. We survey related and potential future work in Sections 8 and 9.
2 Theoretical Foundations
In this section we characterize ct-fuzz’s foundations, including secure information flow, its reduction to traditional safety properties, and coverage-guided greybox fuzzing.
2.1 Secure Information Flow
We consider an abstract notion of secure information flow  over pairs of executions. We say two executions and are -equivalent with respect to some function when . We model program secrets by a declassification function mapping each execution to its declassified content , and say -equivalent executions are comparable.222This notion of declassification is more general than that appearing in the literature . Intuitively, this equivalence relates executions whose input and output values differ solely by secret content, e.g., by the value of a secret key. Similarly, we model attacker capabilities by an observation function mapping each execution to its observable content , and say two -equivalent executions are indistinguishable. Intuitively, this equivalence relates executions which a given observer cannot differentiate, e.g., because of negligible timing differences. In practice, we distinguish timing variations by observing divergences in control-flow decisions and accessed memory locations.
A program is secure when every pair of comparable executions are indistinguishable, for given declassification and observation functions.
2.2 Reducing Security to Safety
Self-composition  is program transformation reducing secure information flow to traditional safety properties. Here we say a given program is safe when it cannot crash. This simple notion of safety can capture violations to any temporal safety property  given adequate program instrumentation. For instance, llvm’s thread, address, and memory sanitizers signal crashes upon thread- and memory-safety violations, and uses of uninitialized memory, respectively [33, 32, 34].
The self-composition is a program (Fig. 2), which executes two copies of a given program in isolation, i.e., execution of each copy is independent of the other, which:
halts execution when the copies’ executions are incomparable, and
crashes when the copies’ executions are both comparable and distinguishable.
Intuitively, safety of the self-composition implies security of the original program. Conversely, if the original is safe and secure, then the self-composition is also safe.
A safe program is secure iff its self-composition is safe .
2.3 Coverage-Guided Greybox Fuzzing
Böhme et al. [10, 9] provide a conceptual overview of coverage-guided greybox fuzzing as implemented by afl-fuzz . For our purposes, a simplistic view (Fig. 2) suffices: fuzzers explore sequences of program executions to expose safety-property violations (manifested as crashes — see §2.2) by randomly mutating the inputs to previously-explored executions according to per-execution feedback. Intuitively, feedback allows the fuzzer to navigate alternate program paths, and in practice, captures basic-blocks transition counts. From the perspective of this work, the salient aspects of afl-fuzz are its broad applicability and efficiency: it works without user interaction and source code, and avoids prohibitive process-creation overheads by sharing executions’ address spaces in a copy-on-write fashion. These features enable the rapid exploration of program executions for arbitrarily-complex software.
3 Design and Implementation Concerns
Applying self-composition (§2.2) to coverage-guided greybox fuzzers (§2.3) poses three basic challenges: efficiently simulating execution pairs; deriving structured-input pairs from fuzzer-provided inputs (fuzz); and detecting leakage.
3.1 Simulating Execution Pairs
The self-composition approach (§2.2) dictates that two identical copies of a program execute in isolation. In the context of testing, achieving isolation includes separating the address spaces of each simulated execution so that the side-effects of one are invisible to the other. On the one hand, the existing approach of duplicating procedures and variables is effective for symbolic analyses like ct-verif . However, actually executing such duplicated programs on their target platforms can alter the originals’ behavior significantly, e.g., due to platform-dependent behavior. On the other hand, executing copies in separate processes undermines the efficacy of fuzzers like afl-fuzz , which largely avoid the overhead of process creation.
3.2 Deriving Structured Inputs from Fuzz
According to secure information flow (§2.1), leaks are only witnessed by comparable executions, i.e., whose declassifications are identical. So on the one hand, testing incomparable executions is wasteful. On the other hand, coverage-guided greybox fuzzers (§2.3) provide inputs (fuzz) by randomly mutating previously-explored inputs. A self-composition which fed fuzz directly to its simulated executions would generate comparable executions with very low likelihood. For instance, when input variable is declassified in the program , the likelihood of randomly generating the fuzz with for comparable and is extremely low for typical datatypes, e.g., 32- and 64-bit integers. It follows that exposing information leaks requires non-trivial transformations from fuzz to structured input pairs.
3.3 Detecting Leakage
According to secure information flow (§2.1), leaks are witnessed by comparable yet distinguishable executions, i.e., whose declassifications are identical, yet observations are distinct. Assuming a reliable mechanism for generating comparable executions (§3.2), monitoring leakage amounts to establishing a notion of observation for a given leakage model, instrumenting the source program to record such observations, and signaling a program crash when observations differ.
4 Software Architecture
We implement self-composition by a lightweight llvm program transformation . We invoke it as part of afl-fuzz’s llvm-based instrumentation  for convenience, before passing the instrumented program to afl-fuzz. The choice of an llvm-based implementation was made for familiarity and convenience, e.g., due to llvm bitcode being typed; in principle, our approach could be implemented at the assembly level. Following the concerns outlined in Section 3, our transformation provides three basic capabilities: efficiently simulating execution pairs, deriving structured inputs-pairs from fuzz, and capturing leakage-relevant observations.
4.1 Efficient Implementation of Self-Composition
To implement self-composition, ct-fuzz borrows the same basic trick that makes afl-fuzz efficient: copy-on-write sharing of address spaces via process forks. Figure 4 sketches our implementation strategy. After initializing the data structures used to capture observations (§4.3), constructing inputs from fuzz (§4.2), and ensuring the preconditions of the program under test, ct-fuzz forks the running process twice. After each child executes the original program on its copy of input, the parent checks equality of the children’s observation traces. Section 5.1 describes the specification of programs, including stating their declassifications and preconditions.
Our fork-based self-composition avoids a potentially-complex address-space management entailed by the existing duplication approach (see §3.1). Forking the existing process creates children with identical address spaces, thus minimizing the potential for artificial divergence. Furthermore, the cost of forking is low on modern operating systems due to copy-on-write optimization: the virtual-memory pages of child processes point to the same physical pages as their parents; pages themselves are only duplicated when either the parent or child dirties them with subsequent writes. While forking provides ample isolation for basic CPU-driven programs like cryptographic primitives, we do not ensure isolation with stateful IO, e.g., interacting with files and sockets; this is a common issue for testing-based approaches.
Our implementation of self-composition also assumes that library functions are deterministic: a sequence of invocations returns identical values in both forked children. Nondeterminism can undermine ct-fuzz’s leakage tests, since divergence between execution pairs may be due to nondeterminism rather than dependence on secrets. This potential is especially apparent in memory allocation: malloc is generally free to return the address of any unallocated chunk of memory. ct-fuzz handles this common case by linking the Jemalloc allocator , which ensures deterministic behavior.
4.2 Deriving Inputs from Fuzz
To transform the randomly-mutated inputs (fuzz) into program inputs which are likely to generate comparable executions (see §3.2), ct-fuzz generates per-program input processors. These processors depend on the signatures of entry points, as well as declassification annotations on program arguments. Besides the program under test, ct-fuzz expects such signatures and annotations to be specified using the API described in Section 5. Given this specification, ct-fuzz constructs an input processor, which constructs program-input pairs by reading (from standard input) one fuzzed instance of each declassified argument, and two fuzzed instances of each secret argument. ct-fuzz invokes both program copies with the same fuzz for declassified arguments, and possibly distinct fuzz for secret arguments. While this mechanism does not guarantee that the corresponding executions are ultimately comparable, since further declassification can occur upon execution, e.g., declassification of output values, it does avoid incomparability due to declassified inputs.333While our current implementation does not handle post-input declassification, this could be done by monitoring declassifications, similarly to monitoring observations — see §4.3.
4.3 Recording Observations and Reporting Leakage
To capture alternate leakage models ct-fuzz provides an extensible mechanism for recording observations. As our initial implementation targets control-flow and cache-based timing leaks, our program transformation inserts instrumentation before branch and memory-access instructions; more precisely, we insert calls to the monitor API of Figure 4. The monitor receives branch-condition values and memory addresses, along with the source location of each instruction, and records observations in a shared memory region for access by parent process. When observations diverge, ct-fuzz signals leakage by inducing a crash which will be reported by the fuzzer; our current implementation causes a segmentation fault by dereferencing address zero.
Our initial prototype provides two distinct implementations of the monitor API. The first constant-time monitor collects traces of branch-condition values and memory addresses directly; executions are distinguishable if they differ on any of the branch-condition values or memory addresses. The second cache-model monitor records Boolean values indicating cache hits or misses in place of memory accesses, allowing for the precise analysis of non-constant-time programs which are nevertheless safe for a given cache architecture, e.g., cache-preloading implementations (see §6). Our prototype uses Sung et al.’s cache model , though any could be used in its place.
To limit the size of the allocated shared-memory region and time for equality-checking, we leverage a fast hash-function  to replace the storage of arbitrarily-long observation sequences with one fixed-size hash value. For each observation , the monitor updates its hash value to for a given hash function . We state the correctness of this optimization as follows, where a perfect hash function is one in which iff .
Given a perfect hash function, two observation sequences are equal iff their lengths and corresponding monitors are equal.
5 Functionality and Capabilities
Our ct-fuzz tool is capable of reporting timing leaks in any code base that can be analyzed with afl-fuzz’s LLVM mode. In this section we demonstrate the use of ct-fuzz on the simple cryptographic function in Figure 5.
5.1 Specifying Argument Preconditions, Declassifications, and Defaults
Our current implementation of ct-fuzz requires a brief specification of the program under test. Specifically, we require information about program arguments: preconditions, declassifications, and default values. Similarly to ct-verif , annotations in ct-fuzz are written directly in the source language (i.e., C/C++), and processed by our instrumentation at compile time. Specifications are attached to entry points using the CT_FUZZ_SPEC and CT_FUZZ_SEED macros. The specification in Figure 5 declassifies the len argument (msg is secret), requires that each byte of the msg buffer be an ascii-encoded decimal digit, and provides a default buffer containing bytes ’1’–’4’.
To facilitate the fuzzing of dynamically-sized buffers like msg, the specification also declares that msg is a buffer of length len at most 4 bytes; alternatively, constant-length buffers can also be specified. Default values provide the initial values (seeds) from which fuzzers begin their mutation-based exploration. While their specification is not strictly required, since fuzzers can begin with initially-empty seeds, reasonable seeds can boost fuzzers’ performance substantially. We provide macros for specifying default values, e.g., SEED_1D_ARR. Similarly, while preconditions are not strictly required for fuzzing, they allow us to isolate leakage-related crashes from safety-related crashes by assuming that the original program is safe, and thus does not crash, so long as the preconditions are met. The provided CT_FUZZ_ASSUME macro triggers exit unless its argument evaluates to true. This mechanism leverages the fuzzer’s ability to recognize branches and automatically synthesize inputs which pass precondition checks.
5.2 Exposure and Diagnosis of Timing Leaks
ct-fuzz applies the self-composition described in §4 on programs with specifications, generating a binary executable and initial seed for fuzzing. The invocation of ct-fuzz:
generates the encrypt binary and seed 0x31 0x32 0x33 0x34 0x04 0x00 0x00 0x00, according to the specification of default argument values. After copying the generated seed to input-dir, afl-fuzz can be invoked directly:444Currently we require manual duplication of the seed for the self-composition’s input pair.
which instantly reports crashes indicating timing leaks, according to the constant-time property. This is the expected result, since the memory locations accessed by the encrypt function depend on the secret contents of msg. Specifically, secrets are used as offsets into the book buffer. In principle, such secret-dependent memory accesses can lead to cache-based timing variations, and ultimately the leakage of msg contents.
To facilitate diagnostics, ct-fuzz includes a mechanism for logging and comparing the observations of the comparable yet distinguishable execution pairs, tracing observations and the source-file locations at which they occur. For example, comparing the traces generated for the leak exposed above:
we spot the divergence due to the dereference of the book buffer on Line 7, column 14.
5.3 Using Alternative Leakage Models
For some applications, the constant-time leakage model is too conservative. For example, since the book buffer in Fig 5 fits into a single cache line (assuming standard modern architectures), it would be unlikely that the secret contents of msg affect timing, since every access to book after the first will hit the cache, independently of msg. Alternate leakage models can be selected in ct-fuzz with the --memory-leakage flag; currently we support two options: address and cache. The latter implements Sung et al.’s model  with fixed values for block size, set associativity, and replacement policy. Extension to parametric and alternative models is straightforward. Using the cache model, we do not discover any timing leaks in the example above.
6 Experience and Case Studies
We have applied ct-fuzz to several popular cryptographic implementations — see §7 for empirical results. To highlight an interesting example, we consider the AES encryption functions of the Botan library invoked in Figure 6. For simplicity, we consider potential leakage of a secret 16-byte key argument, fixing all other parameters.
6.1 Analysis of Constant-Time
Applying ct-fuzz immediately reveals a constant-time violation, witnessed by a pair of inputs differing only by their first byte: 0xaa versus 0x2a. Comparing their execution traces reveals leakage from the SE_word function shown in Figure 7: the single-byte input difference leads to different offsets into the SE table computed by get_byte(3,x).
6.2 Precise Cache Modeling
Although Botan’s AES implementation is not constant time, it is still considered secure against timing leaks due to its use of cache preloading countermeasures. Specifically, every entry in its lookup tables, e.g., the SE table, is accessed before the secret-dependent lookup-table accesses performed during encryption or decryption to ensure that all subsequent secret-dependent accesses hit the cache. However, previous versions were found to be insecure by SC-Eliminator’s static analyzer  due to missing applications of the countermeasure in the aes_key_schedule function.555https://github.com/randombit/botan/commit/09b3d5447d77633d4f9ad0603187ca2a0b017ebd
Applying ct-fuzz to the insecure version immediately reveals a timing leak even with the cache-model monitor. Interestingly, the inputs for the first-reported leak are identical to those reported in the aforementioned constant-time violation; further analysis demonstrates the same source of leakage in Figure 7: one execution’s access of SE[get_byte(3,x)] is a cache miss, while the other’s is a hit. The divergence can be attributed to the fact that the addresses of the four memory accesses in Fig. 7 are proximate for one execution, while for the other execution, the address of the fourth memory access is distant to the previous three, making them belong to different cache lines. As expected, reapplying ct-fuzz to the secure version reveals no timing leaks.
7 Empirical Evaluation
We evaluate ct-fuzz’s ability to uncover timing leaks in a range of cryptographic implementations summarized by Table 1. We analyze multiple entry points from each with the constant-time and cache-model monitors described in Section 4. Besides the benchmarks used to evaluate SC-Eliminator , we have collected several libraries from their sources. Tables 2 and 3 summarize our results.666We experiment on a 3.5GHz Intel i7 Ubuntu 16.04 desktop machine with 16GB DDR3 memory.
For each entry point, we run afl-fuzz 10 times with 10 second timeouts. In cases where afl-fuzz reports crashes, we re-run afl-fuzz 100 times with the same 10 second timeout, and report the mean and standard deviations until crash for: time (in seconds), number of explored executions, and number of program paths explored. Otherwise, we re-run afl-fuzz 2 times with a 100 second time limit, and report the execution and path counts until timeout.
|Implementations taken from original sources|
|BearSSL v0.5: an implementation of the SSL/TLS protocol (RFC 5246) written in C. https://bearssl.org|
|libsodium @973cdb5: a modern, easy-to-use software library for encryption, decryption, signatures, password hashing and more. https://download.libsodium.org/doc|
|OpenSSL v1.1.0h: a robust, commercial-grade, and full-featured toolkit for the Transport Layer Security (TLS) and Secure Sockets Layer (SSL) protocols. https://www.openssl.org|
|poly1305-donna @e6ad6e0: a state-of-the-art message-authentication code. https://github.com/floodyberry/poly1305-donna|
|poly1305-opt @700d5cf: a portable, performant implementation of Poly1305. https://github.com/floodyberry/poly1305-opt|
|s2n @745fdd8: a C99 implementation of the TLS/SSL protocols that is designed to be simple, small, fast, and with security as a priority. https://github.com/awslabs/s2n|
|Implementations sourced from Wu et al. |
|Applied Crypto: textbook implementations from Applied Cryptography: protocols, algorithms, and source code in C .|
|ChronOS: Cryptographic primitives in ChronOS linux. http://www.chronoslinux.org|
|FELICS: lightweight block ciphers for the Internet of Things. https://www.cryptolux.org/index.php/FELICS|
|Libgcrypt: a general purpose cryptographic library originally based on code from GnuPG. https://gnupg.org/software/libgcrypt/index.html|
|SUPERCOP: a toolkit developed by the VAMPIRE lab for measuring the performance of cryptographic software. https://bench.cr.yp.to/supercop.html|
|Botan: a C++ cryptography library released under the permissive Simplified BSD license. https://github.com/randombit/botan/|
|Constant Time||Cache Model|
7.1 Analysis with the Constant-Time Monitor
ct-fuzz swiftly reports constant-time violations due to secret-dependent table lookups, e.g., all of Wu et al.’s benchmarks. OpenSSL’s C implementation of AES encryption leverages substitution boxes. For supposedly constant-time implementations such as BearSSL’s constant-time AES encryption (aes_ct) and libsodium’s constant-time utility functions (sodium_increment, sodium_is_zero), ct-fuzz reports no violations. In our experience with these benchmarks, afl-fuzz does not report any violations, even after several hours, that are not reported within a few seconds. On average, ct-fuzz uncovers violations within half of a second, exploring only hundreds of executions. Execution counts are approximate, since afl-fuzz’s AFL_BENCH_UNTIL_CRASH mode only guarantees termination soon after the first crash. Furthermore, path counts report the number of unique non-crashing control-flow paths discovered by afl-fuzz; paths are often unique since cryptographic implementations tend to use fairly straight-line code.
7.2 Analysis with the Cache-Model Monitor
We further run ct-fuzz with our cache-model monitor (see § 5.3). The cache model is identical to that used in the evaluation of SC-Eliminator : a fully associative LRU cache of 512 64-byte lines. We report no timing leaks for Wu et al.’s FELICS benchmarks  according to the cache-model monitor, which is expected since their lookup tables are small enough to fit into one cache line. This is consistent with SC-Eliminator’s static analysis, although our notion of leakage, framed as a two-safety property, is a more-precise characterization which avoids the potential for false positives — see §8. Furthermore, we report no timing leaks for constant-time functions like aes_ct, which is expected since constant-time is stricter than the absence of leakage according to a precise cache model. The differences in the number of executions explored until crash compared with the constant-time monitor is due to the approximation of execution counts: since the constant-time monitor is more efficient than the cache-model monitor, afl-fuzz squeezes in many more executions between the first crash and termination.
8 Related Work
The literature around secure information flow is vast, and Sabelfeld and Myers offer a compelling survey . Here we highlight recent works to detecting timing side-channel leaks with program analysis. We distinguish between symbolic approaches, e.g., involving verification, model checking, and abstract interpretation, and concrete approaches, e.g., testing and coverage-guided greybox fuzzing, which observe executions on programs’ target platforms. Along another axis, we differentiate works according to how they identify leakage, whether it be tracking the flow of data through program values, statistical measures on the distributions of program executions, or as two-safety properties on pairs of program executions.
8.1 Symbolic Analyses
While our work accurately detects leakage, most related works soundly establish the absence of leaks. In contrast, our testing-based approach sacrifices soundness777While coverage-guided fuzzing is subject to false negatives, e.g., potential leakage not reported, we have found that all potential leakages are swiftly reported in typical cryptographic implementations since control- and data-flow is straightforward. for precision and efficiency, without introducing the potential for false positives or computational bottlenecks due to abstract symbolic reasoning. We identify several works based on symbolic analysis of data-flow or two-safety properties.
8.1.1 Flow Tracking
Many recent works propose static analyses to prove secure information flow. tis-ct  extends Frama-C  with static analysis to compute program dependencies and infer potential leaks. VirtualCert  integrated a type system capable of tracking aliasing and information flow into the CompCert certified compiler . FlowTracker  developed efficient static information-flow analyses based on the single static assignment (ssa) representation rather than program dependence graphs (pdg). Blazy et al.  developed context-sensitive and arithmetic-aware alias analyses to verify constant time of C programs using the Verasco static analyzer . CacheAudit [16, 15] developed effective cache-aware abstract domains to prove the absence of cache-based side channels using abstract interpretation. SC-Eliminator  performs static analysis to identify or prove the absence of potential cache-timing leaks, and synthesizes patches by replacing conditional statements and lookup-table accesses.
Other works employ satisfiability-based techniques. CacheD  uses symbolic execution to identify potential cache-access variations as expressions over secrets. SCInfer  combines type-inference and satisfiability modulo theories (smt) to verify random masking of program secrets. canal  performs analysis-friendly program instrumentation enabling symbolic reasoning about cache-related properties, e.g., which accesses are cache hits. cachefix  uses cbmc  to prove the absence of leaks, or synthesizes patches for discovered leaks.
Barthe et al.  originally proposed the self-composition program transformation to reduce the verification of two-safety properties like secure information flow to traditional safety verification. Terauchi and Aiken  refined their approach with a type-directed transformation to apply self-composition selectively, with a simpler cross-product transformation  around low-security program fragments. Milushev et al.  adapt this approach for use with the klee symbolic execution engine , while ENCoVer  leverages Java PathFinder . Phan  reformulates self-composition as path equivalence for direct symbolic execution of the original program.
Almeida et al.  developed a self-composition based methodology for proving constant-time of C programs using Frama-C . The recently developed ct-verif  and SideTrail  tools apply selective self-composition to verify constant-time and time-balancing, respectively, in C-language cryptographic primitives using the smack verifier . Yang et al.  propose lazy self-composition to apply precise reasoning only as a fallback to coarser techniques like taint analysis. Blazer  proposes quotient partitioning as an alternative, compositional reduction to traditional safety verification.
8.2 Concrete Analyses
More closely aligned with our approach are those based on observing executions on programs’ target platforms, thus avoiding the potential for false positives and computational bottlenecks due to abstract symbolic reasoning. While we consider secure information flow as a two-safety property, other works are based either on tracking data-flow or statistical analysis.
8.2.1 Flow Tracking
The key work in this space is ctgrind , which extends Valgrind  with a taint-tracking mechanism to track whether secrets flow into accessed memory locations. While this avoids the bottlenecks of symbolic reasoning, this approach is susceptible to false positives, since secret-tainted observations are falsely considered leaks when they depend also on public or declassified data values. Considering secure information flow as a two-safety property avoids this source of false positives.
A few approaches detect timing leaks using statistical methods. dudect  records the execution time of many (typically millions) of executions with inputs partitioned into random and fixed values. While measuring execution time directly avoids the imprecision of indirect measurements like memory-access traces, this can also be considered a source of false negatives, since timing observed in a given test environment can vary significantly from other platforms and system loads.
Other statistical approaches measure traces of memory and control-flow operations. data  detects differences among traces of executions with random inputs, before honing in on statistical tests with fixed- versus random-secret inputs; the approach relies on expert interaction for classification of potential leaks, and is susceptible to false positives due to program nondeterminism, e.g., of memory allocators. MicroWalk  proposes alternative statistical tests based on mutual information analysis.
To the best of our knowledge, mutaflow  is the only other concrete-analysis approach considering secure information flow as a two-safety property. Like our work, mutaflow compares observations along two executions in which the second receives the first’s inputs randomly mutated. However, since only program values are observed, mutaflow cannot detect leakage due to side-channels like timing.
9 Conclusion and Future Work
This work demonstrates that testing-based software analysis methodologies can be effectively extended from safety properties to two-safety properties. While our initial application has targeted timing leaks in cryptographic implementations, there are several promising directions for future investigation. Since cryptographic code tends to follow rather simple straight-line control flow, coverage-guided greybox fuzzers offer relatively little over more simplistic random testing; exposing leaks in more complex input-processing applications such as jpeg encoders and decoders  could better exploit two-safety-property fuzzers. Furthermore, ct-fuzz could be applied to other types of leakage, and even other classes of two-safety properties, by extending the instrumentation mechanism; in principle, our approach is capable of exposing violations to any two-safety property expressed as equality between program traces.
This work was funded in part by the US Department of Homeland Security (DHS) Science and Technology (S&T) Directorate under contract no. HSHQDC-16-C-00034. The views and conclusions contained herein are the authors’ and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DHS or the US government.
-  Almeida, J.B., Barbosa, M., Barthe, G., Dupressoir, F., Emmi, M.: Verifying constant-time implementations. In: USENIX Security Symposium. pp. 53–70. USENIX Association (2016)
-  Almeida, J.B., Barbosa, M., Pinto, J.S., Vieira, B.: Formal verification of side-channel countermeasures using self-composition. Sci. Comput. Program. 78(7), 796–812 (2013)
-  Antonopoulos, T., Gazzillo, P., Hicks, M., Koskinen, E., Terauchi, T., Wei, S.: Decomposition instead of self-composition for proving the absence of timing channels. In: PLDI. pp. 362–375. ACM (2017)
-  Athanasiou, K., Cook, B., Emmi, M., MacCarthaigh, C., Schwartz-Narbonne, D., Tasiran, S.: SideTrail: Verifying time-balancing of cryptosystems. In: VSTTE. Lecture Notes in Computer Science, vol. 11294. Springer (2018)
-  Balliu, M., Dam, M., Guernic, G.L.: Encover: Symbolic exploration for information flow security. In: CSF. pp. 30–44. IEEE Computer Society (2012)
-  Barthe, G., Betarte, G., Campo, J.D., Luna, C.D., Pichardie, D.: System-level non-interference for constant-time cryptography. In: ACM Conference on Computer and Communications Security. pp. 1267–1279. ACM (2014)
-  Barthe, G., D’Argenio, P.R., Rezk, T.: Secure information flow by self-composition. Mathematical Structures in Computer Science 21(6), 1207–1252 (2011)
-  Blazy, S., Pichardie, D., Trieu, A.: Verifying constant-time implementations by abstract interpretation. In: ESORICS (1). Lecture Notes in Computer Science, vol. 10492, pp. 260–277. Springer (2017)
-  Böhme, M., Pham, V., Nguyen, M., Roychoudhury, A.: Directed greybox fuzzing. In: ACM Conference on Computer and Communications Security. pp. 2329–2344. ACM (2017)
Böhme, M., Pham, V., Roychoudhury, A.: Coverage-based greybox fuzzing as markov chain. In: ACM Conference on Computer and Communications Security. pp. 1032–1043. ACM (2016)
-  Cadar, C., Dunbar, D., Engler, D.R.: KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: OSDI. pp. 209–224. USENIX Association (2008)
-  Chattopadhyay, S., Roychoudhury, A.: Symbolic verification of cache side-channel freedom. IEEE Trans. on CAD of Integrated Circuits and Systems 37(11), 2812–2823 (2018)
-  Collet, Y.: xxHash: an extremely fast non-cryptographic hash algorithm (2012), http://cyan4973.github.io/xxHash/
-  Cuoq, P., Signoles, J., Baudin, P., Bonichon, R., Canet, G., Correnson, L., Monate, B., Prevosto, V., Puccetti, A.: Experience report: Ocaml for an industrial-strength static analysis framework. In: ICFP. pp. 281–286. ACM (2009)
-  Doychev, G., Köpf, B.: Rigorous analysis of software countermeasures against cache attacks. In: PLDI. pp. 406–421. ACM (2017)
-  Doychev, G., Köpf, B., Mauborgne, L., Reineke, J.: Cacheaudit: A tool for the static analysis of cache side channels. ACM Trans. Inf. Syst. Secur. 18(1), 4:1–4:32 (2015)
-  Evans, J.: jemalloc: a general purpose malloc(3) implementation that emphasizes fragmentation avoidance and scalable concurrency support (2002), http://jemalloc.net
-  Jourdan, J., Laporte, V., Blazy, S., Leroy, X., Pichardie, D.: A formally-verified C static analyzer. In: POPL. pp. 247–259. ACM (2015)
-  Kroening, D.: CBMC: Bounded model checking for software (2001), http://www.cprover.org/cbmc/
-  Langley, A.: ctgrind: Checking that functions are constant time with Valgrind (2010), https://github.com/agl/ctgrind/
Leroy, X.: A formally verified compiler back-end. J. Autom. Reasoning43(4), 363–446 (2009)
-  Manna, Z., Pnueli, A.: Temporal verification of reactive systems - safety. Springer (1995)
-  Mathis, B., Avdiienko, V., Soremekun, E.O., Böhme, M., Zeller, A.: Detecting information flow by mutating input data. In: Software Engineering. LNI, vol. P-279, pp. 61–62. Gesellschaft für Informatik (2018)
-  McLean, J.: A general theory of composition for trace sets closed under selective interleaving functions. In: IEEE Symposium on Security and Privacy. pp. 79–93. IEEE Computer Society (1994)
-  Milushev, D., Beck, W., Clarke, D.: Noninterference via symbolic execution. In: FMOODS/FORTE. Lecture Notes in Computer Science, vol. 7273, pp. 152–168. Springer (2012)
-  Phan, Q.: Self-composition by symbolic execution. In: ICCSW. OASICS, vol. 35, pp. 95–102. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Germany (2013)
-  Rakamaric, Z., Emmi, M.: SMACK: decoupling source language details from verifier implementations. In: CAV. Lecture Notes in Computer Science, vol. 8559, pp. 106–113. Springer (2014)
-  Reparaz, O., Balasch, J., Verbauwhede, I.: Dude, is my code constant time? In: DATE. pp. 1697–1702. IEEE (2017)
-  Rodrigues, B., Pereira, F.M.Q., Aranha, D.F.: Sparse representation of implicit flows with applications to side-channel detection. In: CC. pp. 110–120. ACM (2016)
-  Sabelfeld, A., Myers, A.C.: Language-based information-flow security. IEEE Journal on Selected Areas in Communications 21(1), 5–19 (2003)
-  Schneier, B.: Applied cryptography - protocols, algorithms, and source code in C, 2nd Edition. Wiley (1996)
-  Serebryany, K., Bruening, D., Potapenko, A., Vyukov, D.: Addresssanitizer: A fast address sanity checker. In: USENIX Annual Technical Conference. pp. 309–318. USENIX Association (2012)
-  Serebryany, K., Potapenko, A., Iskhodzhanov, T., Vyukov, D.: Dynamic race detection with LLVM compiler - compile-time instrumentation for threadsanitizer. In: RV. Lecture Notes in Computer Science, vol. 7186, pp. 110–114. Springer (2011)
-  Stepanov, E., Serebryany, K.: Memorysanitizer: fast detector of uninitialized memory use in C++. In: CGO. pp. 46–55. IEEE Computer Society (2015)
-  Sung, C., Paulsen, B., Wang, C.: CANAL: a cache timing analysis framework via LLVM transformation. In: ASE. pp. 904–907. ACM (2018)
-  Team, L.: The LLVM compiler infrastructure (2003), http://llvm.org
-  Terauchi, T., Aiken, A.: Secure information flow as a safety problem. In: SAS. Lecture Notes in Computer Science, vol. 3672, pp. 352–367. Springer (2005)
-  The Valgrind Developers: Valgrind: an instrumentation framework for building dynamic analysis tools (2000), http://www.valgrind.org
-  TrustInSoft: TIS-CT: an adaptation of an existing Frama-C plug-in for computing program dependencies (2011), https://trust-in-soft.com/tis-ct/
-  Visser, W., Pasareanu, C.S., Khurshid, S.: Test input generation with java pathfinder. In: ISSTA. pp. 97–107. ACM (2004)
-  Wang, S., Wang, P., Liu, X., Zhang, D., Wu, D.: Cached: Identifying cache-based timing channels in production software. In: USENIX Security Symposium. pp. 235–252. USENIX Association (2017)
-  Weiser, S., Zankl, A., Spreitzer, R., Miller, K., Mangard, S., Sigl, G.: DATA - differential address trace analysis: Finding address-based side-channels in binaries. In: USENIX Security Symposium. pp. 603–620. USENIX Association (2018)
-  Wichelmann, J., Moghimi, A., Eisenbarth, T., Sunar, B.: Microwalk: A framework for finding side channels in binaries. CoRR abs/1808.05575 (2018)
-  Wu, M., Guo, S., Schaumont, P., Wang, C.: Eliminating timing side-channel leaks using program repair. In: ISSTA. pp. 15–26. ACM (2018)
-  Xu, Y., Cui, W., Peinado, M.: Controlled-channel attacks: Deterministic side channels for untrusted operating systems. In: IEEE Symposium on Security and Privacy. pp. 640–656. IEEE Computer Society (2015)
-  Yang, W., Vizel, Y., Subramanyan, P., Gupta, A., Malik, S.: Lazy self-composition for security verification. In: CAV (2). Lecture Notes in Computer Science, vol. 10982, pp. 136–156. Springer (2018)
-  Zaks, A., Pnueli, A.: Covac: Compiler validation by program analysis of the cross-product. In: FM. Lecture Notes in Computer Science, vol. 5014, pp. 35–51. Springer (2008)
-  Zalewski, M.: american fuzzy lop: a security-oriented fuzzer (2013), http://lcamtuf.coredump.cx/afl/
-  Zhang, J., Gao, P., Song, F., Wang, C.: Scinfer: Refinement-based verification of software countermeasures against side-channel attacks. In: CAV (2). Lecture Notes in Computer Science, vol. 10982, pp. 157–177. Springer (2018)