1 Benchmark Description
VLSAT-1111https://cadp.inria.fr/resources/vlsat is a collection of 100 SAT formulas, which are listed in Table 1. All these formulas are satisfiable and have been designed to accept a large number of models. Each formula is provided as a a separate file, expressed in Conjunctive Normal Form and encoded in the DIMACS-CNF format222http://www.satcompetition.org/2009/format-benchmarks2009.html. Each file is then compressed using bzip2 to save disk space and allow faster downloads. The 100 formulas require 2.1 gigabytes of disk space and 419 megabytes when compressed using bzip2.
2 Scientific Context
These formulas have been generated as a by-product of our recent work  on the decomposition of Petri nets into networks of automata, a problem that has been around since the early 70s. Concretely, we developed a tool chain that takes as input a Petri net (which must be ordinary, safe, and hopefully not too large) and produces as output a network of automata that execute concurrently and synchronize using shared transitions. Precisely, this network is expressed as a Nested-Unit Petri Net (NUPN) , i.e., an extension of a Petri net, in which places are grouped into sets (called units) that denote sequential components. A NUPN provides a proper structuration of its underlying Petri net, and enables formal verification tools to be more efficient in terms of memory and CPU time. Hence, the NUPN concept has been implemented in many tools and adopted by software competitions, such the Model Checking Contest333https://mcc.lip6.fr [7, 6] and the Rigorous Examination of Reactive Systems challenge444http://rers-challenge.org [5, 8]. Each NUPN generated by our tool chain is flat, meaning that its units are not recursively nested in each other, and unit-safe, meaning that each unit has at most one execution token at a time.
Our tool chain works by reformulating concurrency constraints on Petri nets as logical problems, which can be later solved using third-party software, such as SAT solvers, SMT solvers, and tools for graph colouring and finding maximal cliques . We applied our approach to a large collection of more than 12,000 Petri nets from multiple sources, many of which related to industrial problems, such as communication protocols, distributed systems, and hardware circuits. We thus generated a huge collection of SAT formulas, from which we carefully selected a subset of 100 formulas for VLSAT. Figure 1 shows the scalability of our benchmark suite, which properly represents the diversity of our experiments.
3 Structure of VLSAT Formulas
Each of our formulas was produced for a particular Petri net. A formula depends on three factors:
the set of the places of the Petri net;
a concurrency relation defined over such that is the two places and may simultaneously have an execution token; and
a chosen number of units.
A formula expresses whether there exists a partition of into subsets () such that, for each , and for any two places and of , . A model of this formula is thus an allocation of places into units, i.e., a valid decomposition of the Petri net. The value of is chosen large enough so that the formula is satisfiable, i.e., at least one decomposition exists. This can also be seen as an instance of the graph coloring problem, in which colors are to be used for the graph with vertices defined by the places of and edges defined by the concurrency relation.
More precisely, each formula was generated as follows. For each place and each unit , we created a propositional variable that is true iff place belongs to unit . We then added constraints over these variables:
For each unit and each two places and such that and , where is a bijection from places names to the interval , we added the constraint to express that two concurrent places cannot be in the same unit.
For each place , we could have added the constraint to express that belongs to at least one unit, but this constraint was too loose and allowed similar solutions, just by permuting unit names. We thus replaced this constraint by a stricter one that breaks the symmetry between units: for each place , we added the refined constraint , where is a bijection from unit names to the interval .
The VLSAT-1 benchmarks are licensed under the CC-BY Creative Commons Attribution 4.0 International License555License terms available from http://creativecommons.org/licenses/by/4.0.
Experiments presented in this paper were carried out using the Grid’5000666https://www.grid5000.fr testbed, supported by a scientific interest group hosted by Inria and including Cnrs, Renater and several Universities as well as other organizations.
-  Pierre Bouvier, Hubert Garavel, and Hernán Ponce de León. Automatic Decomposition of Petri Nets into Automata Networks – A Synthetic Account. In Ryszard Janicki, Natalia Sidorova, and Thomas Chatain, editors, Proceedings of the 41st International Conference on Application and Theory of Petri Nets and Concurrency (PETRI NETS’20), Paris, France, volume 12152 of Lecture Notes in Computer Science, pages 3–23. Springer, June 2020.
-  Johannes K. Fichte and Markus Hecher. MCC 2020 - Model Counting Competition. https://mccompetition.org/assets/files/2020/MC2020_awards.pdf, July 2020.
-  Johannes K. Fichte, Markus Hecher, and Florim Hamiti. The Model Counting Competition 2020. Technical Report, November 2020.
-  Hubert Garavel. Nested-Unit Petri Nets. Journal of Logical and Algebraic Methods in Programming, 104:60–85, April 2019.
-  Marc Jasper, Maximilian Fecke, Bernhard Steffen, Markus Schordan, Jeroen Meijer, Jaco van de Pol, Falk Howar, and Stephen F. Siegel. The RERS 2017 Challenge and Workshop. In Hakan Erdogmus and Klaus Havelund, editors, Proceedings of the 24th ACM SIGSOFT International SPIN Symposium on Model Checking of Software (SPIN’17), Santa Barbara, CA, USA, pages 11–20. ACM, July 2017.
-  Fabrice Kordon, Hubert Garavel, Lom Messan Hillah, Emmanuel Paviot-Adet, Loïg Jezequel, Francis Hulin-Hubard, Elvio Amparore, Marco Beccuti, Bernard Berthomieu, Hugues Evrard, Peter G. Jensen, Didier Le Botlan, Torsten Liebke, Jeroen Meijer, Jiří Srba, Yann Thierry-Mieg, Jaco van de Pol, and Karsten Wolf. MCC’2017 – The Seventh Model Checking Contest. Transactions on Petri Nets and Other Models of Concurrency, XIII:181–209, 2018.
-  Fabrice Kordon, Hubert Garavel, Lom Messan Hillah, Emmanuel Paviot-Adet, Loïg Jezequel, César Rodríguez, and Francis Hulin-Hubard. MCC’2015 – The Fifth Model Checking Contest. Transactions on Petri Nets and Other Models of Concurrency, XI:262–273, 2016.
-  Bernhard Steffen, Marc Jasper, Jeroen Meijer, and Jaco van de Pol. Property-Preserving Generation of Tailored Benchmark Petri Nets. In Proceedings of the 17th International Conference on Application of Concurrency to System Design (ACSD’17), Zaragoza, Spain, pages 1–8. IEEE Computer Society, June 2017.