A sample size heuristic for network scale-up studies

11/18/2021
by   Nathaniel Josephs, et al.
0

The network scale-up method (NSUM) is a survey-based method for estimating the number of individuals in a hidden or hard-to-reach subgroup of a general population. In NSUM surveys, sampled individuals report how many others they know in the subpopulation of interest (e.g. "How many sex workers do you know?") and how many others they know in subpopulations of the general population (e.g. "How many bus drivers do you know?"). NSUM is widely used to estimate the size of important epidemiological risk groups, including men who have sex with men, sex workers, HIV+ individuals, and drug users. Unlike several other methods for population size estimation, NSUM requires only a single random sample and the estimator has a conveniently simple form. Despite its popularity, there are no published guidelines for the minimum sample size calculation to achieve a desired statistical precision. Here, we provide a sample size formula that can be employed in any NSUM survey. We show analytically and by simulation that the sample size controls error at the nominal rate and is robust to some forms of network model mis-specification. We apply this methodology to study the minimum sample size and relative error properties of several published NSUM surveys.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 20

11/25/2020

Thirty Years of The Network Scale up Method

Estimating the size of hard-to-reach populations is an important problem...
11/23/2021

Bayesian Sample Size Prediction for Online Activity

In many contexts it is useful to predict the number of individuals in so...
05/12/2021

Sample size planning for pilot studies

Pilot studies are often the first step of experimental research. It is u...
09/21/2021

A Correlated Network Scale-up Model: Finding the Connection Between Subpopulations

Aggregated relational data (ARD), formed from "How many X's do you know?...
10/26/2016

Estimating the Size of a Large Network and its Communities from a Random Sample

Most real-world networks are too large to be measured or studied directl...
11/04/2021

On provision of UK neighbourhood population statistics beyond 2021

Census 2021 may well be the last of its kind in the UK. For provision of...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Estimating the size of a hidden or hard-to-reach population is an important problem in demography, epidemiology, and public health research [10]. Traditional methods, such as capture-recapture [56, 27, 40, 28, 43, 20, 8, 7, 2] and benchmark multiplier methods [44, 20, 61, 62] require multiple independent samples, but it can be costly to obtain multiple samples and difficult to guarantee that samples collected under different designs are independent [1]. Bernard et al. [4] introduced the network scale-up method (NSUM) to estimate the number of people who died in the 1985 Mexico City earthquake. NSUM has subsequently been employed to estimate the number of men who have sex with men [13, 58, 52, 15], sex workers [32, 23, 24, 15], trafficked persons [49], infected or at-risk individuals [30, 29, 54, 46, 50, 9, 15, 22, 25, 53, 19, 18, 45], drug users [26, 39, 48, 15, 32], prisoners [63], victims of disasters [5], abortions [41], choking incidents in children [51], religious individuals [60], and people in one’s personal network [35, 38].

As Maltiel et al. [33]

summarized, “NSUM is based on the idea that for all individuals, the probability of knowing someone in a given subpopulation is the size of that subpopulation divided by the overall population size.” In the basic NSUM

[6, 37, 31, 17], investigators obtain a single random sample of individuals in the general population, who are not necessarily members of the hidden population. Each respondent reports the number of others they know in the general population (or the number they know in several subgroups of the general population so that the number of others they know in the general population may be estimated) and also the number they know in the target population. The ratio of these average counts is multiplied by the known size of the general population to estimate the size of the target population.

Population size estimation using NSUM relies on several assumptions related to homogeneity of the underlying network of acquaintanceships and the accuracy of reported counts. Several variants and generalizations of NSUM extend the method to accommodate more flexible assumptions using complex estimators. For example, McCormick and Zheng [36] provide an adjustment for recall bias, in which the number of contacts is underestimated in large groups and overestimated in small groups. Habecker et al. [16] and Feehan and Salganik [14] formalize the incorporation of unequal sampling weights for surveyed individuals into NSUM. Maltiel et al. [33] propose a Bayesian approach that models recall bias, transmission error, and barrier effects directly. Feehan and Salganik [14] introduce a method that generalizes NSUM (called G-NSUM) by recognizing that in an undirected network, total in-degrees and out-degrees must be equal, a fact that can be exploited when two samples are obtainable. The G-NSUM was further generalized for venue-based sampling [57]. While these methods offer improved and more efficient estimates, the classic NSUM method remains widely used due to its simplicity.

As NSUM surveys grow in popularity, practitioners need guidance on how to design NSUM studies to achieve accurate and reliable results. Researchers have noted that population-size estimates from NSUM can have high variance

[6, 32, 35, 46]. This may be due in part to the lack of a coherent framework that guides investigators in choosing a sample size for empirical studies. To address this gap, we present a sample size heuristic that enables researchers to calculate the number of respondents needed to estimate the size of a hidden population at a given relative error. We investigate the properties of the sample size heuristic and analyze its performance under network model mis-specificiation. Finally, we perform a retrospective sample size analysis for several existing NSUM studies and conclude with recommendations for empirical NSUM study design.

2 Setting and NSUM estimator

Consider a set of individuals of known size called the general population. A subset of individuals of size comprise the hidden population; we wish to estimate . NSUM relies on information about the network of relationships between individuals. The meaning of these relationships varies according to the type of study, but may include social, epidemiological, or communication relationships. Let represent the general population network, where is the set of individuals in the general population and is the set of relevant relationships between population members. We assume is undirected and simple, that is, contains no parallel edges or self-loops. Then for all , we have whenever and share the relationship of interest. Next, define to be the number of connections to the hidden population of person , that is, the number of links between person and members of the hidden population . Finally, define the personal network size of person – which is also called ’s degree – as , the number of edges incident to person in . For simplicity, we assume below that is directly observed, but in practice NSUM studies often estimate using the known population method [29, 30] or the summation method [34].

The statistical performance of the NSUM estimator relies on assumptions about the distribution of personal network sizes and reported connections to the hidden population that imply certain global features of the underlying general population network. In many cases, these assumptions correspond to a particular random graph model. Laga et al. [31] provides a summary of network distributional assumptions in the NSUM literature. In what follows, we assume the general population network has Erdős-Rényi distribution [12]. This framework can be extended to any vertex-exchangeable graph model [11], and we illustrate in Section 4.1, with details in Appendix C, that the sample size heuristic is robust to violations of vertex-exchangeability.

Under an Erdős-Rényi model for the network , edge relationships exist between individuals in , and between individuals in and , with probability independently of other relationships. Consequently, the following marginal degree models for person hold:

(1)

It follows that

If is a member of the target population, then and .

Solving for gives

(2)

This yields an immediate formula for in terms of the expected general and hidden population degrees:

(3)

To estimate the hidden population size , consider a random sample of individuals from , each of whom reports how many individuals they know in different subgroups of the general population (which allows us to estimate ) and how many individuals they know in the hidden population (which is ). Using this information, NSUM estimates both of the expectations in (3

) empirically by averages. Then a method of moments equation for (

2) is

Solving for , we arrive at the classic NSUM estimator [30],

(4)

3 Sample size heuristic

Investigators conducting an NSUM survey must determine the minimum number of samples to collect in order to obtain a suitably precise estimate of the hidden population size . To do this, investigators must first specify how precise they want their estimates to be. This means choosing a value for two parameters: (i) the confidence level, ; and (ii) the relative margin of error,

. In many applied studies, researchers choose to use 95% confidence intervals to express sampling uncertainty in their estimates, meaning that

. The relative margin of error, , describes how close, in relative terms, the estimate is expected to be to the population value [55]. For example, a relative margin of error of 10% () and a confidence level of 95% () means that researchers want a sample size that produces estimates within 10% of the true value at least 95% of the time (that is, in at least 95% of repeated samples).

Formally, the minimum sample size is the smallest such that

(5)

where is the relative absolute error of the estimate. In other words, investigators seek the smallest sample size that ensures the relative error is small with high probability.

Heuristic 1.

Let the relative margin of error be , the confidence level be for , the prevalence of the hidden population be , and the average personal network size be . The minimum sample size needed to satisfy (5) is then

(6)

where is the ceiling operator and is the quantile of the standard normal distribution.

In order to use the sample size formula (6), researchers must specify the values for three population quantities: (i) the size of the general population, ; (ii) the prevalence of the hidden population, ; and (iii) the average size of respondents’ personal networks, . Several properties of the minimum sample size can be deduced immediately from (6), and these properties can be helpful in guiding the choice of these three parameters. First, when the general population size is large, the contribution of to the minimum sample size in (6) is negligible. Second, the consequences of error in estimates of and are evident. If the prevalence of the hidden population is under-estimated, the minimum sample size increases; if is over-estimated, the minimum sample size increases. Therefore, it is preferable to over-estimate the prevalence to ensure a conservative sample size. If the average population degree is under-estimated, the minimum sample size increases; if is over-estimated, the minimum sample size decreases. Therefore it may be preferable for investigators seeking a conservative sample size to use an under-estimate of the average general population degree.

Figure 1, shows the relationship between prevalence and average degree in terms of the minimum sample size for a confidence level of 95% (), a relative margin of error of 10% (), and a general population size of . For example, when , we have , and thus if the relative margin of error is , the prevalence is and the average degree is , then the minimum sample size from (6) is approximately

Figure 1: Sample size as a function of prevalence and average degree for . We truncate the minimum sample size at the population size .

Heuristic 1 is appropriate when subjects are sampled uniformly at random from the population. In practice, NSUM surveys may employ more complex sampling designs using multi-stage sampling, clustering, and stratification. These complex design features affect the variance of estimates. One common approach to modifying the sample size heuristic in (6) is through the use of a design effect for quantifying the impact of a complex sampling design on an estimate. It is defined as the ratio of the variances of an estimator with a given sampling design to a simple random sample; if is an NSUM estimator with a complex sampling design, then

(7)

If the design effect is known a priori, then the minimum sample size calculation from Heuristic 1 can be multiplied by it to obtain an adjusted estimate.

Heuristic 2.

Let be the known design effect for a given sampling scheme. The minimum sample size needed to achieve (5) is approximately

(8)

Alternatively, once a sample has been collected, an effective sample size can be computed as .

4 Results

4.1 Simulation studies

We first assess the relative error and coverage rates using the minimum sample size by simulation. We employ a factorial design that varies the population size, the prevalence, the nominal levels, and the underlying population graph model. We include i) an Erdős-Rényi network [12], ii) an exponential random graph [42], iii) a preferential attachment network [3], iv) a stochastic block model [21], and v) a small-world network [59]. Each graph model has the same expected density around , which allows us to isolate the effect of the graph topology and assess the robustness to network model mis-specification, i.e. how well our minimum sample size calculation works when the underlying population graph implies degree models that violate (1).

The details and results are left to Appendix 4.1, but the conclusions are as follows. The average relative error is below the tolerated level for all values of and when the underlying graph model is ER, ERGM, and small-world. For low prevalence, PA and SBM have average relative errors that exceed the tolerance level, but this is mitigated as prevalence and population size increase. Similarly, the average coverage is conservative for ER, ERGM, and small-world across different nominal levels , whereas it suffers for small populations with low prevalence for PA and SBM. As expected, the error bars are larger for larger values of . Finally, the relative error and coverage drift smoothly away from the desired levels as the underlying graph model deviates from the assumption of vertex-exchangeability.

4.2 Case studies

We now conduct a retrospective sample size analysis of published empirical NSUM surveys. Each published study reports the population size , the sample size employed in the study, and the population size estimate . To derive information about the marginal degree model, we assume that the degree reports and follow (1),

(9)

where , , and are the estimates from the study. In other words, we take the published estimate as the true hidden population size . From this information, we calculate the minimum sample sizes according to (6) using the implied average population degree and prevalence given the published value of the general population size . We use a relative error tolerance of 10% () and 95% confidence level ().

Using these parameters, we compute 10,000 Monte Carlo NSUM estimates of the target population size. That is, in each replicate, we sample degree reports and following the Binomial models in (9), and use them to calculate the NSUM estimate of . For each replicate, we compute the relative error of our NSUM estimate compared to the published estimate in the study, and report the Monte Carlo average relative error over all 10,000 replicates, which we denote RelErr. We apply this retrospective sample size calculation to seven published NSUM studies.

First, we revisit a study by Killworth et al. [30] to estimate the number of HIV+ individuals in 1994 in the US, around the height of the AIDS crisis. Using a telephone survey, the authors randomly sampled members of the US population. Despite the above-average response rate, the authors recognized the potential for some non-response bias. However, they do not discuss why this sample size was chosen, but they report that the survey cost $6.50 per respondent and took 10 minutes to conduct, which suggests there may have been resource constraints. One of the major contributions of this study is an improved technique for estimating personal network sizes, along with one of the first reports of the distribution of network sizes for a random sample of the US population. Using these estimates, the authors concluded that a 95% confidence interval for the number of HIV+ individuals was , which was in line with estimates from the CDC on seroprevalence.

Second, we look at two applications of NSUM to estimate the number of sex workers in China [15, 24]. Chongqing, the largest province in China, had its first reported HIV case in 1993. By 2011, there were nearly 12,000 reported cases, yet population estimates of key affected populations were then unknown. Citing the need for targeted interventions and resource allocation, Guo et al. [15] employed a multistage random sample of 2,957 individuals in order to obtain NSUM estimates of key populations including female sex workers (FSW). Using their survey results, the authors estimated , which had not previously been estimated in Chongqing. This led to a 95% confidence interval of for the number of FSW. They also found that sex workers have a high respect factor, but, while they argue that this suggests their NSUM estimates are less likely to be underestimated, their estimate when adjusting for respect factor was 10% lower. Although this is an important study, being one of the first applications of NSUM in China, the authors note several limitations that ultimately bring additional uncertainty to the conclusions.

More recently, Jing et al. [24] incorporated a randomized response technique to an NSUM survey in 2012 of 7,964 individuals in Taiyuan, China. The authors estimate the FSW population to be , which was similar to the more expensive multiplier methods and consistent with previously reported estimates of sex worker prevalence in Asia. The authors argue that this demonstrates the the appropriateness of their adjusted NSUM approach.

Next, we look at two applications of NSUM to estimate the number of men who have sex with men (MSM) [13, 58]. Since 2008, the number of HIV cases in Japan has risen constantly, with 89% of new cases attributed to MSM. However, prior to 2012, the size of the MSM population in Japan had not been estimated “in a rigorous manner” [13]. To address this, Ezoe et al. [13] employed the first internet-based NSUM estimator. The authors surveyed 1,500 individuals who were registered to Intage, an internet-research agency. They estimated the MSM prevalence among the male population to be 2.87%, which was comparable to previous studies using the direct-estimation method. These results suggest that internet surveys can be combined with NSUM for an even quicker and lower-cost method, especially in stigmatized populations.

Wang et al. [58] are also interested in estimating the number of MSM, but in Shanghai, China. Instead of an internet-based survey, the authors conducted a community-based survey across 19 districts in Shanghai consisting of 3,907 participants. Using NSUM, the authors report a 95% confidence interval for as , but they note that “we did not take the sample design into consideration when performing variance estimation.”

Finally, we consider two applications of NSUM to quantify drug use [16, 19]. In the first, Heydari et al. [19] are interested in estimating the number of individuals using methadone maintenance therapy (MMT) in Kerman, Iran. Such data, which was previously not available, is needed to assess the effectiveness of MMT. The authors used two cross-sectional studies with multi-stage sampling to recruit 2,550 individuals. Using NSUM, they report . Furthermore, they were able to use this to estimate the treatment failure ratio, which was a novel application of NSUM and further evidence of its widespread applicability.

Similarly, Habecker et al. [16] use NSUM to estimate the number of heroin users in Nebraska in the past 30 days. In 2014, the authors received 550 completed mail surveys, which included additional questions that allowed the authors to compute and incorporate sampling weights into an improved NSUM estimator. Such improvements are particularly important when knowledge of the scope of the problem is time-sensitive. Ultimately, this yielded an estimate of a 95% confidence interval of for the number of heroin users in the past 30 days, which may be a proxy for the number of current heroin users.

Hidden population RelErr
Heroin users in Nebraska [16] 550 1,879,321 368 604 0.118 0.02 3,383
FSW in Taiyuan, China [24] 7,964 3,454,927 3,866 137 0.15 0.03 2,610
MMT users in Kerman, Iran [19] 2,550 611,401 5,289 235 2.03 0.01 197
FSW in Chongqing, China [15] 2,957 28,000,000 31,576 311 0.077 0.78 1,141
MSM in Shanghai, China [58] 3,907 24,000,000 36,354 236 0.159 0.56 1,119
HIV+ individuals in US [30] 1,554 250,000,000 800,000 286 0.91 0.02 438
MSM in Japan [13] 1,500 62,348,977 1,789,416 174 5.09 0.02 81
Table 1: Retrospective analysis for average relative error over 10,000 Monte Carlo estimates and minimum sample sizes from (6). The population size and average degrees are estimates from the given NSUM study.
Figure 2: Study sample size compared to the minimum sample size from (6) calculated based on each study’s reported estimates of prevalence and average network size. Most of the points lie below the line , suggesting that the actual sample size was larger than necessary to achieve 10% precision with 95% confidence.

Table 1 shows the results of the retrospective analysis on these seven studies. Figure 2 compares the actual study sample sizes and the retrospectively calculated minimum sample sizes using (6). While most of the studies have an average relative error below , Guo et al. [15] and Wang et al. [58] may have much larger relative errors. In both of these cases, the study sample size is larger than our minimum sample size, but the average hidden population degree is much smaller than the average general population degree times the prevalence. This suggests the topology of the hidden population network differs from that of the general population network, which was a specific limitation listed by Guo et al. [15].

5 Discussion

In this paper, we have presented a simple heuristic for the minimum sample size that controls relative error (6). The heuristic is easy to employ and theoretically justified under the assumption that the general population network follows the Erdős-Rényi random graph model. Furthermore, simulations show that the sample size heuristic is robust to a variety of deviations from this idealized setting.

We also demonstrated how published NSUM studies have sample sizes that scale similarly to this minimum sample size, though most have used a sample size larger than our retrospectively calculated minimum sample size, implying that they could have saved resources by administering surveys to fewer respondents. That said, since the minimum sample sizes are functions of the reported estimates of prevalence and average network degrees, which are all quite high, it is possible that this is a propagation of biased estimates. On the other hand, one of the studies has , which implies they may have low precision in their estimates of the sizes of the populations of interest. In general, practitioners appear to be intuiting our heuristic as many of the study sample sizes are on the same order of magnitude as our minimum sample sizes.

In order to use the formula (6), investigators must specify values for , , and . In some cases, estimates of and may be based on the results of previous studies. But, if little existing information about the hidden population is available, our results show that it will be conservative to err on the side of assuming that the hidden population is relatively low in prevalence (low ) and that personal network sizes are relatively small (low ).

While our heuristic should help practitioners design future NSUM studies, there are several limitations to our analysis. First, the relative error controlled for in Heuristic 1 may not provide the desired absolute precision. Second, the normal approximation of our standardized estimator may be invalid for small samples. Third, sample size formula may differ for more complex sampling designs and the design effects in Heuristic 2 for adjusting the minimum sample size may be unknown. Finally, non-sampling error, for instance from imperfect reporting, may dominate statistical error leading to invalid estimates even for in some cases.

Acknowledgements

We are grateful to Si Cheng, Jinghao Sun, and Clifford Zinnes for helpful comments. This work was supported by the Eunice Kennedy Shriver National Institute of Child Health and Development (1DP2HD091799-01 and P2C HD 073964 via the Berkeley Population Center). DMF also thanks the Berkeley Center for the Economics and Demography of Aging, which is funded by the National Institute on Aging (5P30AG012839).

References

  • [1] A. S. Abdul-Quader, A. L. Baughman, and W. Hladik (2014) Estimating the size of key populations: current status and future possibilities. Current Opinion in HIV and AIDS 9 (2), pp. 107–114. Cited by: §1.
  • [2] N. T. J. Bailey (1951) On estimating the size of mobile populations from recapture data. Biometrika 38 (3/4), pp. 293–306. Cited by: §1.
  • [3] A. Barabási and R. Albert (1999) Emergence of scaling in random networks. Science 286 (5439), pp. 509–512. Cited by: Appendix C, §4.1.
  • [4] H. R. Bernard, E. C. Johnsen, P. D. Killworth, and S. Robinson (1991) Estimating the size of an average personal network and of an event subpopulation: some empirical results. Social Science Research 20 (2), pp. 109–121. Cited by: §1.
  • [5] H. R. Bernard, P. D. Killworth, E. C. Johnsen, G. A. Shelley, and C. McCarty (2001) Estimating the ripple effect of a disaster. Connections 24 (2), pp. 18–22. Cited by: §1.
  • [6] H. R. Bernard, T. Hallett, A. Iovita, E. C. Johnsen, R. Lyerla, C. McCarty, M. Mahy, M. J. Salganik, T. Saliuk, O. Scutelniciuc, G. A. Shelley, P. Sirinirund, S. Weir, and D. F. Stroup (2010) Counting hard-to-count populations: The network scale-up method for public health. Sexually Transmitted Infections 86 (Suppl 2), pp. ii11–15. Cited by: §1, §1.
  • [7] D. Böhning, B. Suppawattanabodee, W. Kusolvisitkul, and C. Viwatwongkasem (2004) Estimating the number of drug users in Bangkok 2001: a capture-recapture approach using repeated entries in one list. European Journal of Epidemiology 19 (12), pp. 1075. Cited by: §1.
  • [8] M. Bouchard (2007) A capture-recapture model to estimate the size of criminal populations and the risks of detection in a marijuana cultivation industry. Journal of Quantitative Criminology 23 (3), pp. 221–241. Cited by: §1.
  • [9] R. B. Center (2012) Estimating the size of key populations at higher risk of HIV through a household survey (ESPHS). Technical report RBC/IHDPC, SPF, UNAIDS and ICF International, Calverton, Maryland, USA. Cited by: §1.
  • [10] S. Cheng, D. J. Eck, and F. W. Crawford (2020) Estimating the size of a hidden finite set: large-sample behavior of estimators. Statistics Surveys 14, pp. 1–31. Cited by: §A.3, §A.3, §1.
  • [11] P. Diaconis and S. Janson (2007) Graph limits and exchangeable random graphs. arXiv preprint arXiv:0712.2749. Cited by: §2.
  • [12] P. Erdős and A. Rényi (1959) On random graphs I. Publicationes Mathematicae 6, pp. 290–297. Cited by: Appendix C, §2, §4.1.
  • [13] S. Ezoe, T. Morooka, T. Noda, M. L. Sabin, and S. Koike (2012) Population size estimation of men who have sex with men through the network scale-up method in Japan. PLoS One 7 (1), pp. e31184. Cited by: §1, §4.2, Table 1.
  • [14] D. M. Feehan and M. J. Salganik (2016) Estimating the size of hidden populations using the generalized network scale-up estimator. Sociological Methodology 46 (1), pp. 153–186. Cited by: §1.
  • [15] W. Guo, S. Bao, W. Lin, G. Wu, W. Zhang, W. Hladik, A. Abdul-Quader, M. Bulterys, S. Fuller, and L. Wang (2013) Estimating the size of HIV key affected populations in Chongqing, China, using the network scale-up method. PLoS One 8 (8), pp. e71796. Cited by: §1, §4.2, §4.2, Table 1.
  • [16] P. Habecker, K. Dombrowski, and B. Khan (2015) Improving the network scale-up estimator: incorporating means of sums, recursive back estimation, and sampling weights. PLoS One 10 (12), pp. e0143406. Cited by: §1, §4.2, §4.2, Table 1.
  • [17] P. Habecker (2017) Who do you know: improving and exploring the network scale-up method. Ph.D. Thesis, The University of Nebraska-Lincoln. Cited by: §1.
  • [18] A. A. Haghdoost, M. R. Baneshi, S. Haji-Maghsoodi, H. Molavi-Vardanjani, and E. Mohebbi (2015) Application of a network scale-up method to estimate the size of population of breast, ovarian/cervical, prostate and bladder cancers. Asian Pacific Journal of Cancer Prevention 16 (8), pp. 3273–3277. Cited by: §1.
  • [19] Z. Heydari, M. R. Baneshi, H. Sharifi, M. Zamanian, S. Haji-Maghsoudi, and F. Zolala (2019) Evaluation of the treatment failure ratio in individuals receiving methadone maintenance therapy via the network scale up method. International Journal of Drug Policy 73, pp. 36–41. Cited by: §1, §4.2, Table 1.
  • [20] M. Hickman, V. Hope, L. Platt, V. Higgins, M. Bellis, T. Rhodes, C. Taylor, and K. Tilling (2006) Estimating prevalence of injecting drug use: a comparison of multiplier and capture–recapture methods in cities in England and Russia. Drug and Alcohol Review 25 (2), pp. 131–140. Cited by: §1.
  • [21] P. W. Holland, K. B. Laskey, and S. Leinhardt (1983) Stochastic blockmodels: first steps. Social networks 5 (2), pp. 109–137. Cited by: Appendix C, §4.1.
  • [22] A. JafariKhounigh, A. A. Haghdoost, S. SalariLak, A. H. Zeinalzadeh, R. Yousefi-Farkhad, M. Mohammadzadeh, and K. Holakouie-Naieni (2014) Size estimation of most-at-risk groups of hiv/aids using network scale-up in Tabriz, Iran. Journal of Clinical Research & Governance 3 (1), pp. 21–26. Cited by: §1.
  • [23] M. A. Jami, M. Baneshi, and M. Nasirian (2021) Population size estimation of high-risk behavior in Isfahan, Iran: using the network scale-up method in 2018. Journal of Biostatistics and Epidemiology 7 (2), pp. 120–130. Cited by: §1.
  • [24] L. Jing, Q. Lu, Y. Cui, H. Yu, and T. Wang (2018) Combining the randomized response technique and the network scale-up method to estimate the female sex worker population size: an exploratory study. Public Health 160, pp. 81–86. Cited by: §1, §4.2, §4.2, Table 1.
  • [25] L. Jing, C. Qu, H. Yu, T. Wang, and Y. Cui (2014) Estimating the sizes of populations at high risk for hiv: a comparison study. PLoS One 9 (4), pp. e95601. Cited by: §1.
  • [26] C. Kadushin, P. D. Killworth, H. R. Bernard, and A. A. Beveridge (2006) Scale-up methods as applied to estimates of heroin use. Journal of Drug Issues 36 (2), pp. 417–440. Cited by: §1.
  • [27] M. Karami, S. Khazaei, J. Poorolajal, A. Soltanian, and M. Sajadipoor (2017) Estimating the population size of female sex worker population in Tehran, Iran: application of direct capture–recapture method. AIDS and Behavior 27 (8), pp. 1–7. Cited by: §1.
  • [28] B. Khan, H. Lee, and K. Dombrowski (2018) One-step estimation of networked population size with anonymity using respondent-driven capture-recapture and hashing. PLoS One 13 (4), pp. e0195959. Cited by: §1.
  • [29] P. D. Killworth, E. C. Johnsen, C. McCarty, G. A. Shelley, and H. R. Bernard (1998) A social network approach to estimating seroprevalence in the United States. Social Networks 20 (1), pp. 23–50. Cited by: §1, §2.
  • [30] P. D. Killworth, C. McCarty, H. R. Bernard, G. A. Shelley, and E. C. Johnsen (1998) Estimation of seroprevalence, rape, and homelessness in the United States using a social network approach. Evaluation Review 22 (2), pp. 289–308. Cited by: §A.3, §1, §2, §2, §4.2, Table 1.
  • [31] I. Laga, L. Bao, and X. Niu (2021) Thirty years of the network scale-up method. Journal of the American Statistical Association (just-accepted), pp. 1–33. Cited by: §1, §2.
  • [32] A. Maghsoudi, M. R. Baneshi, M. Neydavoodi, and A. Haghdoost (2014) Network scale-up correction factors for population size estimation of people who inject drugs and female sex workers in Iran. PLoS One 9 (11), pp. e110917. Cited by: §1, §1.
  • [33] R. Maltiel, A. E. Raftery, T. H. McCormick, and A. J. Baraff (2015) Estimating population size using the network scale up method. The Annals of Applied Statistics 9 (3), pp. 1247. Cited by: §A.3, Appendix A, §1, §1.
  • [34] C. McCarty, P. D. Killworth, H. R. Bernard, E. C. Johnsen, and G. A. Shelley (2001) Comparing two methods for estimating network size. Human organization 60 (1), pp. 28–39. Cited by: §2.
  • [35] T. H. McCormick, M. J. Salganik, and T. Zheng (2010) How many people do you know?: Efficiently estimating personal network size. Journal of the American Statistical Association 105 (489), pp. 59–70. Cited by: §1, §1.
  • [36] T. H. McCormick and T. Zheng (2007) Adjusting for recall bias in “how many x’s do you know?” surveys. In Proceedings of the Joint Statistical Meetings, Cited by: §1.
  • [37] T. H. McCormick (2020) The network scale-up method. The Oxford Handbook of Social Networks, pp. 153. Cited by: §A.3, §1.
  • [38] S. Narouee, M. Shatti, M. Didevar, and M. Nasehi (2020) Estimating social network size using network scale-up method (nsum) in Iranshahr, Sistan and Baluchestan Province, Iran. Medical journal of the Islamic Republic of Iran 34, pp. 35. Cited by: §1.
  • [39] A. Nikfarjam, M. Shokoohi, A. Shahesmaeili, A. A. Haghdoost, M. R. Baneshi, S. Haji-Maghsoudi, A. Rastegari, A. A. Nasehi, N. Memaryan, and T. Tarjoman (2016) National population size estimation of illicit drug users through the network scale-up method in 2013 in Iran. International Journal of Drug Policy 31, pp. 147–152. Cited by: §1.
  • [40] G. Paz-Bailey, J. Jacobson, M. Guardado, F. Hernandez, A. Nieto, M. Estrada, and J. Creswell (2011) How many men who have sex with men and female sex workers live in El Salvador? Using respondent-driven sampling and capture-recapture to estimate population sizes. Sexually Transmitted Infections 87 (4), pp. 279–282. Cited by: §1.
  • [41] A. Rastegari, M. R. Baneshi, S. Haji-Maghsoudi, N. Nakhaee, M. Eslami, H. Malekafzali, and A. A. Haghdoost (2014) Estimating the annual incidence of abortions in Iran applying a network scale-up approach. Iranian Red Crescent Medical Journal 16 (10). Cited by: §1.
  • [42] G. Robins, P. Pattison, Y. Kalish, and D. Lusher (2007) An introduction to exponential random graph (p*) models for social networks. Social networks 29 (2), pp. 173–191. Cited by: Appendix C, §4.1.
  • [43] S. C. Robles, L. D. Marrett, E. A. Clarke, and H. A. Risch (1988) An application of capture-recapture methods to the estimation of completeness of cancer registration. Journal of Clinical Epidemiology 41 (5), pp. 495–501. Cited by: §1.
  • [44] A. Safarnejad, N. T. Nga, and V. H. Son (2017) Population size estimation of men who have sex with men in Ho Chi Minh City and Nghe An using social app multiplier method. Journal of Urban Health 94 (3), pp. 339–349. Cited by: §1.
  • [45] H. Sajjadi, Z. Jorjoran Shushtari, M. Shati, Y. Salimi, M. Dejman, M. Vameghi, S. Karimi, and Z. Mahmoodi (2018) An indirect estimation of the population size of students with high-risk behaviors in select universities of medical sciences: a network scale-up study. PLoS One 13 (5), pp. e0195364. Cited by: §1.
  • [46] M. J. Salganik, D. Fazito, N. Bertoni, A. H. Abdo, M. B. Mello, and F. I. Bastos (2011) Assessing network scale-up estimates for groups most at risk of HIV/AIDS: evidence from a multiple-method study of heavy drug users in Curitiba, Brazil. American Journal of Epidemiology 174 (10), pp. 1190. Cited by: §1, §1.
  • [47] H. Seltman (2012) Approximations for mean and variance of a ratio. Unpublished note. Cited by: §A.1.
  • [48] K. Sheikhzadeh, M. R. Baneshi, M. Afshari, and A. A. Haghdoost (2016) Comparing direct, network scale-up, and proxy respondent methods in estimating risky behaviors among collegians. Journal of Substance Use 21 (1), pp. 9–13. Cited by: §1.
  • [49] J. F. Shelton (2015) Proposed utilization of the network scale-up method to estimate the prevalence of trafficked persons. In Forum on Crime and Society, Vol. 8, pp. 85–94. Cited by: §1.
  • [50] M. Shokoohi, M. R. Baneshi, and H. Ali-akbar (2012) Size estimation of groups at high risk of HIV/AIDS using network scale up in Kerman, Iran. International Journal of Preventive Medicine 3 (7). Cited by: §1.
  • [51] S. Snidero, B. Morra, R. Corradetti, and D. Gregori (2007) Use of the scale-up methods in injury prevention research: an empirical assessment to the case of choking in children. Social Networks 29 (4), pp. 527–538. Cited by: §1.
  • [52] L. Sulaberidze, A. Mirzazadeh, I. Chikovani, N. Shengelia, N. Tsereteli, and G. Gotsadze (2016) Population size estimation of men who have sex with men in tbilisi, georgia; multiple methods and triangulation of findings. PLoS One 11 (2), pp. e0147413. Cited by: §1.
  • [53] A. K. J. Teo, K. Prem, M. I. Chen, A. Roellin, M. L. Wong, H. H. La, and A. R. Cook (2019) Estimating the size of key populations for hiv in singapore using the network scale-up method. Sexually Transmitted Infections 95 (8), pp. 602–607. Cited by: §1.
  • [54] UNAIDS and World Health Organization (2010) Guidelines on estimating the size of populations most at risk to HIV. Technical report Geneva, Switzerland. External Links: Link Cited by: §1.
  • [55] R. Valliant, J. A. Dever, and F. Kreuter (2013) Practical tools for designing and weighting survey samples. Vol. 1, Springer. Cited by: §3.
  • [56] P. G. van der Heijden, I. de Vries, D. Böhning, and M. Cruyff (2015) Estimating the size of hard-to-reach populations using capture-recapture methodology, with a discussion of the International Labour Organization’s global estimate of forced labour. In Forum on Crime and Society, Vol. 8, pp. 109–136. Cited by: §1.
  • [57] A. M. Verdery, S. Weir, Z. Reynolds, G. Mulholland, and J. K. Edwards (2019) Estimating hidden population sizes with venue-based sampling: extensions of the generalized network scale-up estimator. Epidemiology (Cambridge, Mass.) 30 (6), pp. 901. Cited by: §1.
  • [58] J. Wang, Y. Yang, W. Zhao, H. Su, Y. Zhao, Y. Chen, T. Zhang, and T. Zhang (2015) Application of network scale up method in the estimation of population size for men who have sex with men in Shanghai, China. PLoS One 10 (11), pp. e0143118. Cited by: §1, §4.2, §4.2, §4.2, Table 1.
  • [59] D. J. Watts and S. H. Strogatz (1998) Collective dynamics of ‘small-world’ networks. Nature 393 (6684), pp. 440–442. Cited by: Appendix C, §4.1.
  • [60] X. Y. Yang and F. Yang (2017) Estimating religious populations with the network scale-up method: a practical alternative to self-report. Journal for the Scientific Study of Religion 56 (4), pp. 703–719. Cited by: §1.
  • [61] D. Zhang, L. Wang, F. Lv, W. Su, Y. Liu, R. Shen, and P. Bi (2007) Advantages and challenges of using census and multiplier methods to estimate the number of female sex workers in a Chinese city. AIDS Care 19 (1), pp. 17–19. Cited by: §1.
  • [62] D. Zhang, F. Lv, L. Wang, L. Sun, J. Zhou, W. Su, and P. Bi (2007) Estimating the population of female sex workers in two Chinese cities on the basis of the HIV/AIDS behavioural surveillance approach combined with a multiplier method. Sexually Transmitted Infections 83 (3), pp. 228–231. Cited by: §1.
  • [63] T. Zheng, M. J. Salganik, and A. Gelman (2006) How many people do you know in prison? Using overdispersion in count data to estimate social structure in networks. Journal of the American Statistical Association 101 (474), pp. 409–423. Cited by: §1.

Appendix A Variance approximations

In order to find the smallest that satisfies (5), we first need to compute the variance of the NSUM estimator in (4). Under the marginal degree model in (1), if we assume is fixed, then we have

(10)

Although early NSUM methods similarly treated these degrees as fixed, Maltiel et al. [33] argues that they should be treated as random. We show that (10) is a good approximation of the variance even when

is random. In particular, we show it is a conservative estimator, i.e. at least as big as the variance approximations under alternative degree models. To do so, we begin with a review of two methods for approximating the variance of a ratio of random variables.

a.1 Taylor expansion for moments

If is a random variable, then we can approximate its moments by taking the expectation of an approximation of [47]. Let and . Similarly, let and . Then the first-order approximation of the variance is

(11)

Note that this approach exploits the linearity of expectation, but there is no guarantee that inclusion of higher moments will improve the estimates. For example, the first-order approximation of is

(12)

whereas the second-order approximation is

which, from simulation, is usually not an improvement on (12).

a.2 Variance decomposition

For a model that specifies the conditional distribution of , or a marginal model whose induced conditonal model is tractable, we can use the law of total variance so that we only need to linearize :

(13)

a.3 Examples

One drawback of the model in (1) is that it does not guarantee that . Instead, by treating as random, it can be shown [10, Supplementary] that with general population sampling (from ), (1) implies

Alternatively, a conditional degree model can be specified to ensure via

(14)

Killworth et al. [30] use the following variant of (14) without specifying a model for :

(15)

Under (15), (4) is the MLE, which is unbiased with variance [33, 10, 37].

In Table 2, we provide variance approximations under different degree models. Recall that our variance estimate in (10) under the model in (1) is

This is always greater than the variance estimate for model (1) in the first row using linearization and they are equal as . Similarly, as and , the variance estimates using the decomposition for the models in rows two and three converge to twice the estimate in (10). For these reasons, the variance in (10) is conservative.

Model Linearize Linearize
)
Table 2: Variance estimates under under two approximations: linearizing , explained in Section A.1, and linearizing , explained in Section A.2.

Appendix B Derivation of Heuristic 1

Recall that we want to find the smallest such that

(16)

By the central limit theorem, when

is large, is approximated by a mean-zero normal distribution whose variance we recall from (10) as

By symmetry, (16) becomes

Dividing by , we have

and thus by normality,

where

is the cumulative distribution function of a standard normal random variable. It follows that

where is the quantile of the standard normal distribution. Substituting the definition of , we find that

and therefore the minimum sample size is

Appendix C Simulations

We assess the relative error and coverage rates using the minimum sample size, by simulation. We employ a factorial design that varies the population size (), the prevalence (), the nominal levels (), and the underlying population graph model. We include i) an Erdős-Rényi network [12], , ii) an exponential random graph [42] with coefficients of for edges and triangles, iii) a preferential attachment network [3] with power of and new edges at each step, iv) a stochastic block model [21],

and v) a small-world network [59] with a lattice of 50. We refer to these five random graph models as ER, ERGM, PA, SBM, and small-world, respectively. Each graph model has the same expected density around , which allows us to isolate the effect of the graph topology and assess the robustness to network model mis-specification, i.e. how well our minimum sample size calculation works when the underlying population graph implies degree models that violate (1).

Figure 3 shows the average relative error and coverage over 500 replicates. The columns represent the size of the population, , and the rows represent the different (true) underlying population graph model. In the top plot, the average relative error is below the tolerated level for all values of and when the underlying graph model is ER, ERGM, and small-world. For low prevalence, PA and SBM have average relative errors that exceed the tolerance level, but this is mitigated as prevalence and population size increase. Results are similar in the bottom plot: the average coverage is conservative for ER, ERGM, and small-world across different nominal levels , whereas it suffers for small populations with low prevalence for PA and SBM. As expected, the error bars are larger for larger values of .

Figure 3: Relative error of and confidence interval coverage of the true over 500 simulated NSUM studies, as a function of prevalence and different values of the confidence level . (Top) Average relative error with the dashed line representing the relative error threshold . (Bottom) Average coverage with the dashed line representing the nominal levels

. In both plots, error bars are one standard deviation. Black panels indicate infeasibility of computation under ERGM when

.

Although the minimum sample size works well for all of the graph models, we would like to better understand how the performance changes as the underlying population network gradually deviates away from an Erdős-Rényi network. To do so, we consider three models. First, we have a preferential attachment (PA) network in which of the nodes are an Erdős-Rényi subgraph. Second, we have a two-block stochastic block model (SBM) with on the diagonal and off the diagonal. Finally, we have an exponential random graph model with a triangle coefficient of , which we denote ERGM . For all of these models, we vary from 0 to 1, and, as before, the average density for all of the networks is fixed at . The results are shown in Figure 4. The error and coverage lie above and below, respectively, the tolerated levels only with extreme deviation from vertex-exchangeability, which further highlights the robustness of our estimator.

Figure 4: Relative error of and confidence interval coverage of the true over 500 simulated NSUM studies, by random network model. (Top) Average relative error with the dashed line representing the tolerance . (Bottom) Average coverage with the dashed line representing the nominal level . In both plots, error bars are one standard deviation.