Estimating Residential Broadband Capacity using Big Data from M-Lab

by   Xiaohong Deng, et al.

Knowing residential broadband capacity profiles across a population is of interest to both consumers and regulators who want to compare or audit performance of various broadband service offerings. Unfortunately, extracting broadband capacity from speed tests in public datasets like M-Lab is challenging because tests are indexed by client IP address which can be dynamic and/or obfuscated by NAT, and variable network conditions can affect measurements. This paper presents the first systematic effort to isolate households and extract their broadband capacity using 63 million speed test measurements recorded over a 12 month period in the M-Lab dataset. We first identify a key parameter, the correlation between measured speed and congestion count for a specific client IP address, as an indicator of whether the IP address represents a single house, or a plurality of houses that may be dynamically sharing addresses or be aggregated behind a NAT. We then validate our approach by comparing to ground truth taken from a few known houses, and at larger scale by checking internal consistency across ISPs and across months. Lastly, we present results that isolate households and estimate their broadband capacity based on measured data, and additionally reveal insights into the prevalence of NAT and variations in service capacity tiers across ISPs.


page 1

page 5

page 7

page 8


A Comparative Analysis of Ookla Speedtest and Measurement Labs Network Diagnostic Test (NDT7)

Consumers, regulators, and ISPs all use client-based "speed tests" to me...

Nautilus: A Framework for Cross-Layer Cartography of Submarine Cables and IP Links

Submarine cables constitute the backbone of the Internet. However, these...

Comparing Broadband ISP Performance using Big Data from M-Lab

Comparing ISPs on broadband speed is challenging, since measurements can...

Modelling of IP Geolocation by use of Latency Measurements

IP Geolocation is a key enabler for many areas of application like Conte...

Estimating the Node Degree of Public Peers and Detecting Sybil Peers Based on Address Messages in the Bitcoin P2P Network

Some peers in the Bitcoin P2P network distributed a huge amount of spam ...

Dragoon: Advanced Modelling of IP Geolocation by use of Latency Measurements

IP Geolocation is a key enabler for many areas of application like deter...

Virtual Ground Truth, and Pre-selection of 3D Interest Points for Improved Repeatability Evaluation of 2D Detectors

In Computer Vision, finding simple features is performed using classifie...

Please sign up or login with your details

Forgot password? Click here to reset