A New Algorithm for Two-Stage Group Testing

01/20/2019
by   Ilya Vorobyev, et al.
0

Group testing is a well-known search problem that consists in detecting of s defective members of a set of t samples by carrying out tests on properly chosen subsets of samples. In classical group testing the goal is to find all defective elements by using the minimal possible number of tests in the worst case. In this work, two-stage group testing is considered. Using the hypergraph approach we design a new search algorithm, which allows improving the known results for fixed s and t→∞. For the case s=2 this algorithm achieves information-theoretic lower bound 2_2t(1+o(1)) on the number of tests. Also, the problem of finding m out of s defectives is considered.

READ FULL TEXT VIEW PDF

Authors

page 1

page 2

page 3

page 4

01/22/2020

Optimal Multistage Group Testing Algorithm for 3 Defectives

Group testing is a well-known search problem that consists in detecting ...
11/15/2019

Optimal adaptive group testing

The group testing problem is concerned with identifying a small number k...
07/16/2020

Community aware group testing

Group testing pools together diagnostic samples to reduce the number of ...
01/25/2021

On the binary adder channel with complete feedback, with an application to quantitative group testing

We determine the exact value of the optimal symmetric rate point in the ...
02/27/2018

Follow Up on Detecting Deficiencies: An Optimal Group Testing Algorithm

In a recent volume of Mathematics Magazine (Vol. 90, No. 3, June 2017) t...
02/10/2022

Group Testing on General Set-Systems

Group testing is one of the fundamental problems in coding theory and co...
05/27/2020

Group testing with nested pools

We iterate Dorfman's pool testing algorithm <cit.> to identify infected ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Group testing problem was introduced by Dorfman in [1]. Suppose that we have a population of items, some of which are defective. Our task is to find all defective items by performing a minimal number of tests. The test is carried out on a properly chosen subset (pool) of the set of samples. The test outcome is positive if the tested set contains at least one defective element; otherwise, it is negative.

In group testing, two types of algorithms are usually considered. In adaptive group testing, at each step the algorithm decides which group to test by observing the responses of the previous tests. In non-adaptive algorithm, all tests are carried out in parallel. Multistage algorithm is a compromise solution to the group testing problem. In -stage algorithms all tests are divided into stages. Tests from the th stage may depend on the outcomes of the tests from the previous stages.

Define to be the minimal worst-case total number of tests needed to find all defective members of a set of samples using at most stages. Also define the optimal rate of -stage search algorithm as

By the similar way we define the rate of fully adaptive algorithms.

In many applications, it is much cheaper and faster to perform tests in parallel, but non-adaptive algorithms require far more tests than adaptive ones. More precise, for non-adaptive algorithms it is known [2, 3] that . In contrast, adaptive algorithms allow to achieve the rate . Rather surprisingly, for 2-stage algorithms it was proved that tests are already sufficient [4, 5, 6]. This fact emphasizes the importance of multistage algorithms.

I-a Previous results

We refer the reader to the monographs [7, 8] for a survey on group testing and its applications.

For non-adaptive algorithms the best known asymptotic () lower [6] and upper [9] bounds are as follows

In addition, we refer to the work [10], where the best lower and upper bounds on were established

For the case of -stage algorithms, , the only known upper bound is information-theoretic one

(1)

The best asymptotic () lower bound for 2-stage algorithms was established in [9]

(2)

The best result for the case was obtained in [11]

All of the lower bounds mentioned above were probabilistic. We want to refer to 2 constructive lower bounds for the case . In [12] the authors obtained a 2-stage algorithm with rate . In [13] an explicit 4-stage testing scheme with rate was constructed. This bound matches the information-theoretic upper bound, i.e. the presented 4-stage algorithms allow to achieve the same rate as a fully adaptive algorithm.

The aim of this work is a further development of the bounds on the rates .

I-B Outline

In Section II, we introduce the notation and describe the hypergraph approach to group testing problem. In Section III, we establish Theorem 1 which states . This result means that 2-stage testing schemes can achieve the same rate as fully adaptive algorithms for . Theorem 4 proved in Section IV is a generalization of Theorem 1 for the case of arbitrary number of defectives. Numerical results and comparison with the best previously known bounds are presented in Table I. In section V, we consider the problem of finding out of defective elements. Theorem 5 shows that we can find defective elements with the rate . Finally, Section VI concludes the paper.

Ii Preliminaries

Throughout the paper we use and for the number of elements and defectives, respectively. By we denote the set . The binary entropy function is defined as usual

A binary -matrix with rows and columns

is called a binary code of length and size . The number of ’s in the codeword , i.e., , is called the weight of , and parameter , , is the relative weight. The quantity is called the rate of the code .

Represent non-adaptive tests with a binary matrix in the following way. An entry equal if and only if th element is included in th test. Let denote the disjunctive sum of binary columns . For any subset

define the binary vector

which later will be called the outcome vector. By , , denote an unknown set of defects.

In the sequel, we consider 2-stage search algorithms. During the first stage, some pools are tested in parallel. Tests for the second stage depend on the outcomes of the first stage.

Let us describe the general idea of our approach. Suppose that we use a binary matrix at the first stage. As a result of performed tests we get the outcome vector . Construct a hypergraph in the following way. The set of vertexes coincides with the set of samples . The set of edges consists of all sets , , such that . In other words, the set of edges of the hypergraph represents all possible defective sets of size . We want to construct a code for the first stage such that for any possible outcome vector the hypergraph has a constant number of edges. In that case, we need to test only the constant amount of non-isolated vertexes individually to find all defective elements. The asymptotic() rate of such scheme is equal to the asymptotic rate of the code .

Iii Algorithm for 2 defectives

We apply the plan described in the previous section to the case . Note that in this case we have graph instead of hypergraph .

Theorem 1.

As it was mentioned before, this bound matches the bound for a fully adaptive algorithm. In addition, the algorithm from this theorem can be used to find not only defectives but also at most defectives. But to keep things simple, here we consider only the case of exactly 2 defectives.

Proof of Theorem 1.

Consider a random matrix

of size , each columns of which is chosen independently and uniformly from the set off all columns of weight , . To keep the notation simple, we ignore the fact that may not be an integer. Fix the constant and consider a graph , where vector is from .

Call the index a -bad index of the first type if the degree of the vertex in the graph is at least . Call the index a -bad index of the second type if the vertex is included in some matching of cardinality at least in the graph . Recall that matching is a set of edges without common vertexes. Finally, call the index a bad index if there exists a vector such that is a -bad index of the first or the second type.

The following two propositions imply the theorem.

Proposition 1.

If the maximum vertex degree and the maximum cardinality of a matching in a graph don’t exceed , then .

Proposition 2.

For any and , there exists an integer such that for the mathematical expectation of the number of bad indexes less than for big enough.

Let us show how the Theorem 1 can be deduced from these two propositions. Indeed, take the parameters , , from Proposition 2. Thus, for big enough there exists a matrix without bad indexes. Use that matrix as a testing matrix at the first stage. There is no bad indexes, therefore, for any outcome vector graph does not contain a matching of size greater than or a vertex with a degree greater than . Applying the proposition 1 to the graph , we conclude that it has at most edges. We finish the search procedure by testing all non-isolated vertexes of the graph individually.

Now we prove propositions 1 and 2.

Proof of Proposition 1.

Fix an arbitrary maximum matching , , in the graph . Denote the set of endpoints of as , . Since is a maximum matching, every edge has at least one endpoint in the set . Therefore, the total number of edges is upper bounded by

Proof of Proposition 2.

The probability that a fixed index

is a -bad index of the first type for some outcome vector of weight can be upper bounded by

where is a probability that for some index the equation holds.

The probability that a fixed index is a -bad index of the second type for some outcome vector of weight can be upper bounded by

where is a probability that for some indexes the equation holds true.

Therefore, the mathematical expectation of the number of bad indexes can be upper bounded as follows

(3)

Take such that , . Then the mathematical expectation of the number of bad indexes is less than

To finish the proof of the proposition it is sufficient to show that and for all . It is easy to see that , hence it is enough to verify the inequality .

Taking the logarithm and dividing by , we obtain

(4)

For the maximal value of the left-hand side is equal to , therefore, the inequality holds.

The theorem is proved. ∎

Iv Algorithm for defectives

To describe an algorithm for finding defectives we must introduce some new notions. Fix an integer and consider a -uniform hypergraph . Call the set of edges a -bad configuration if , , for any and . In other words, -bad configuration consists of edges such that the intersection of every two edges is the same set of size . Call a code a -good code, , if the hypergraph doesn’t contain a -bad configuration for any outcome vector and integer . Let be the minimal length of -good code of size . The asymptotic rate of -good code is defined as follows

(5)

Denote the limit by .

The following lemma demonstrates the connection between -good codes and two-stage group testing problem.

Lemma 2.
(6)
Proof of Lemma 2.

Use -good code of size as a test matrix at the first stage. Then for any outcome vector hypergraph doesn’t contain -bad configurations for .

Proposition 3.

If a -uniform hypergraph doesn’t contain -bad configurations for , then the number of edges is at most , where doesn’t depend on .

Proof of Proposition 3.

Suppose, seeking a contradiction, that a hypergraph without bad configurations contains more than edges. Exact formula for will be specified later. Construct a complete graph , which vertex set corresponds to the edges of hypergraph . Color the edge of the graph in color , if the cardinality of the intersection is equal to , .

Recall that Ramsey number is a minimal integer such that if the edges of a complete graph are colored with different colors, then for some between and , the graph must contain a complete subgraph of size whose edges are all color . Here we need only the fact that the number exists.

Take . Then for some there exists a set of edges , …, from the hypergraph such that for any .

Consider an edge . Any other edge from the set has common vertexes with . Taking , we obtain that some vertexes belong to another edges from the set . But then the set of edges forms bad configuration of type . This contradiction proves the proposition. ∎

Using Proposition 3 we conclude that the hypergraph has at most edges. Thus, we can find all defectives by testing all non-isolated vertices individually at the second stage. The number of tests at the second stage doesn’t depend on the number of elements , therefore, taking limits and we obtain the inequality (6).

To obtain a lower bound on the rate we use a random coding method.

Lemma 3.

Define a function

(7)

where is a unique root of the equation

(8)

Define as follows.

(9)

where

(10)
(11)

Then

(12)

The proof of this lemma can be found in the Appendix.

Lemma 3 and Lemma 2 give us

Theorem 4.
(13)

The best lower bounds for the case are given by disjunctive list-decoding codes with the length of the list  [9]. In Table I we compare bounds given by Theorem 4 with the best previously known lower bounds.

3 4 5 6
old 0.199 0.145 0.114 0.094
new 0.3219 0.199 0.145 0.114
TABLE I: Comparison of old and new lower bounds on the rate

Note that the new lower bound for defective elements coincides with the old lower bound for defective elements. Putting into the condition (11) we obtain the expression . The same expression appears in the derivations of the old lower bound. It means that new algorithm doesn’t improve the previous best known bound for .

V Finding out of defectives

The technique developed in the previous sections can be used to find only part of the defectives. Suppose that we want to find only out of defectives. Define to be the minimal worst-case total number of tests needed to find out of defective members in a set of samples using at most stages. Also define the optimal rate of -stage search algorithm as

The problem of finding out of defectives with the help of non-adaptive algorithms was formulated in [14] for and in [15] for the general case. The best results were obtained in [16, 17], where the following bound was proved

for some constants and .

For two-stage algorithms, we don’t know if it is possible to find all defectives with the rate . But it turns out that we can find at least half of the defectives with this rate.

Theorem 5.
(14)
Proof of Theorem 5.

This proof is based on the following technical lemma.

Lemma 6.
(15)

The proof of this lemma is postponed to the Appendix. Let us show how it implies the theorem. Use -good code of size as a test matrix at the first stage. Then for any outcome vector hypergraph doesn’t contain -bad configurations for . Form subset of edges such that for every the intersection has at most vertices, and for every , the intersection has at least vertices. Such subset can be constructed greedily by adding edges one by one while it is possible. At the second stage we test all non-isolated vertices of the hypergraph . At least vertices of every edge will be tested by construction of , thus, we will find at lest defectives. The number of tests at the second stage is upper bounded by . The hypergraph doesn’t contain any -bad configurations for . Using proposition 3 we conclude that . Therefore, the number of tests at the second stage doesn’t depend on the total number of elements . Taking limits and and using lemma 6 we obtain the desired inequality (14). ∎

Vi Conclusion

A new algorithm for two-stage group testing was proposed, which improves previously known results. For the case of 2 defectives, this algorithm has the optimal rate 0.5. Also, a two-stage algorithm which finds at least half of the defectives with the rate was constructed.

Development of the algorithm, which will achieve the optimal rate for the number of defectives greater than 2, is a natural open problem. Another interesting task is to obtain an upper bound on the rate , , which is stronger than information-theoretic bound .

We note that the technique used in this paper could be also applied to another group testing models, such as, for example, symmetric or threshold group testing.

Vii Acknowledgement

I. Vorobyev was supported in part by RFBR through grant nos. 18-07-01427 A, 18-31-00361 MOL_A.

References

  • [1] R. Dorfman, “The detection of defective members of large populations,” The Annals of Mathematical Statistics, vol. 14, no. 4, pp. 436–440, 1943.
  • [2] A. G. D’yachkov and V. V. Rykov, “Bounds on the length of disjunctive codes,” Problemy Peredachi Informatsii, vol. 18, no. 3, pp. 7–13, 1982.
  • [3] M. Ruszinkó, “On the upper bound of the size of the r-cover-free families,” Journal of Combinatorial Theory, Series A, vol. 66, no. 2, pp. 302–310, 1994.
  • [4] A. De Bonis, L. Gasieniec, and U. Vaccaro, “Optimal two-stage algorithms for group testing problems,” SIAM Journal on Computing, vol. 34, no. 5, pp. 1253–1270, 2005.
  • [5] A. Rashad, “Random coding bounds on the rate for list-decoding superimposed codes,” Problems of Control and Information Theory, vol. 19, no. 2, pp. 141–149, 1990.
  • [6] A. G. D’yachkov, “Lectures on designing screening experiments,” arXiv preprint arXiv:1401.7505, 2014.
  • [7] D. Du, F. K. Hwang, and F. Hwang, Combinatorial group testing and its applications.   World Scientific, 2000, vol. 12.
  • [8] F. Cicalese, Fault-Tolerant Search Algorithms, ser. Monographs in Theoretical Computer Science. An EATCS Series.   Springer Berlin Heidelberg, 2013.
  • [9] A. G. D’yachkov, I. V. Vorob’ev, N. Polyansky, and V. Y. Shchukin, “Bounds on the rate of disjunctive codes,” Problems of Information Transmission, vol. 50, no. 1, pp. 27–56, 2014.
  • [10] D. Coppersmith and J. B. Shearer, “New bounds for union-free families of sets,” the electronic journal of combinatorics, vol. 5, no. 1, p. 39, 1998.
  • [11] P. Damaschke, A. S. Muhammad, and G. Wiener, “Strict group testing and the set basis problem,” Journal of Combinatorial Theory, Series A, vol. 126, pp. 70–91, 2014.
  • [12] P. Damaschke and A. S. Muhammad, “A toolbox for provably optimal multistage strict group testing strategies,” in International Computing and Combinatorics Conference.   Springer, 2013, pp. 446–457.
  • [13] A. G. D’yachkov, I. V. Vorobyev, N. Polyanskii, and V. Y. Shchukin, “On a hypergraph approach to multistage group testing problems,” in Information Theory (ISIT), 2016 IEEE International Symposium on.   IEEE, 2016, pp. 1183–1191.
  • [14] M. Csros and M. Ruszinkó, “Single-user tracing and disjointly superimposed codes,” IEEE transactions on information theory, vol. 51, no. 4, pp. 1606–1611, 2005.
  • [15] B. Laczay and M. Ruszinkó, “Multiple user tracing codes,” in Information Theory, 2006 IEEE International Symposium on.   IEEE, 2006, pp. 1900–1904.
  • [16] N. Alon and V. Asodi, “Tracing a single user,” European Journal of Combinatorics, vol. 27, no. 8, pp. 1227–1234, 2006.
  • [17] ——, “Tracing many users with almost no rate penalty,” IEEE transactions on information theory, vol. 53, no. 1, pp. 437–439, 2007.
  • [18] A. G. D’yachkov, I. V. Vorob’ev, N. Polyansky, and V. Y. Shchukin, “Almost disjunctive list-decoding codes,” Problems of Information Transmission, vol. 51, no. 2, pp. 110–131, 2015.