Measuring HTTP/3: Adoption and Performance

02/24/2021
by   Martino Trevisan, et al.
0

The third version of the Hypertext Transfer Protocol (HTTP) is currently in its final standardization phase by the IETF. Besides better security and increased flexibility, it promises benefits in terms of performance. HTTP/3 adopts a more efficient header compression schema and replaces TCP with QUIC, a transport protocol carried over UDP, originally proposed by Google and currently under standardization too. Although HTTP/3 early implementations already exist and some websites announce its support, it has been subject to few studies. In this work, we provide a first measurement study on HTTP/3. We testify how, during 2020, it has been adopted by some of the leading Internet companies such as Google, Facebook and Cloudflare. We run a large-scale measurement campaign toward thousands of websites adopting HTTP/3, aiming at understanding to what extent it achieves better performance than HTTP/2. We find that adopting websites often host most web page objects on third-party servers, which support only HTTP/2 or even HTTP/1.1. Our experiments show that HTTP/3 provides sizable benefits only in scenarios with high latency or very poor bandwidth. Despite the adoption of QUIC, we do not find benefits in case of high packet loss, but we observe large diversity across website providers' infrastructures.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 3

07/07/2020

Third-party transfers in WLCG using HTTP

Since its earliest days, the Worldwide LHC Computational Grid (WLCG) has...
04/04/2020

An Early Benchmark of Quality of Experience Between HTTP/2 and HTTP/3 using Lighthouse

Google's QUIC (GQUIC) is an emerging transport protocol designed to redu...
10/20/2020

Exploring HTTPS Security Inconsistencies: A Cross-Regional Perspective

If two or more identical HTTPS clients, located at different geographic ...
10/27/2021

Sharding and HTTP/2 Connection Reuse Revisited: Why Are There Still Redundant Connections?

HTTP/2 and HTTP/3 avoid concurrent connections but instead multiplex req...
04/04/2018

Co Hijacking Monitor: Collaborative Detecting and Locating Mechanism for HTTP Spectral Hijacking

With the rapid growth of mobile internet, mobile application, like websi...
04/26/2019

Characterizing web pornography consumption from passive measurements

Web pornography represents a large fraction of the Internet traffic, wit...
05/11/2019

HSTS Preloading is Ineffective as a Long-Term, Wide-Scale MITM-Prevention Solution: Results from Analyzing the 2013 - 2017 HSTS Preload List

HSTS (HTTP Strict Transport Security) serves to protect websites from ce...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

The Hypertext Transfer Protocol (HTTP) is the king of web protocols and is used to access a plethora of services available on the Internet, from websites to social networks and collaborative platforms. HTTP was born in the early 90s, and its first version 1.1 was standardized in 1997 [5]. For more than a decade, this version remained unchanged. Only in 2014, HTTP/2 [1] was proposed, with substantial changes in the framing mechanisms. HTTP/3 is the third version of HTTP and is currently in the final standardization phase at the IETF [2]. It promises performance benefits and security improvements compared to version 2. As a major change, HTTP/3 replaces TCP as transport layer in favor of QUIC, a UDP-based protocol originally proposed by Google and currently being standardized by the IETF [7]. Furthermore, it introduces a more effective header compression mechanism and exploits TLS 1.3 [13] or higher to improve the level of security.

HTTP/3 is expected to supplant HTTP/2 in the next years, and some of the leading Internet companies already announced its support during 2020 (e.g., the CloudFlare CDN111https://blog.cloudflare.com/http3-the-past-present-and-future/ and Facebook222https://engineering.fb.com/2020/10/21/networking-traffic/how-facebook-is-bringing-quic-to-billions/). However, currently neither the real state of deployment nor the performance benefits of HTTP/3 have been measured yet.

In this paper, we fill this gap by running a large-scale measurement study. Firstly, we collect data to understand to what extent the web ecosystem already adopted HTTP/3. Secondly, we run an experimental campaign to measure the benefits introduced by HTTP/3 on the users’ Quality of Experience (QoE), and how HTTP/3 helps in different network conditions.

Using the open-source HTTPArchive Dataset, we find thousands of websites supporting HTTP/3, most of them owned by a handful of Internet hyper-giants, i.e., Facebook, Google, and Cloudflare. We automatically visit the HTTP/3 websites under diverse network conditions to measure the performance benefits in terms of QoE-related metrics. We visit

websites while emulating artificial latency, packet loss, and limiting the bandwidth. In total, we run visits over a period of one month. We find that HTTP/3 benefits emerge only on particular network conditions and strongly differ across website providers. Our key findings are the following:

  • Google, Facebook and Cloudflare are the early adopters of HTTP/3, representing almost the totality of currently HTTP/3 enabled websites.

  • For most of them, the majority of web page objects are still hosted on non-HTTP/3 third-party servers.

  • We observe sizable performance benefits only in scenarios with high latency or poor bandwidth.

  • The performance speed-up largely depends on the infrastructure hosting the website.

  • Websites requiring few connections to load are those benefiting the most.

The remainder of the paper is organized as follows: Section II describes HTTP/3 and illustrates related work. Section III presents the datasets we use and how we collect them, and Section IV illustrates our results on HTTP/3 adoption and performance. Finally, Section V states the limitations of our measurements and poses the basis for future work, while Section VI concludes the paper.

Ii Background

Ii-a Http/3

Fig. 1: Protocol stack for different HTTP versions.

The HTTP/3 protocol is the third version of the well-known Hypertext Transfer Protocol, born in the 90s to transfer multimedia content and hyper-textual documents over the nascent Internet. With its version 1.1, it has been the king of web protocols for more than 20 years, superseded only by its second version HTTP/2 in 2014. HTTP/2 implemented several novel features, especially to improve how the data is framed and transported. It promised to make the web faster even if its benefits have not universally accepted [16, 12].

HTTP/3 is the third version of HTTP. It is currently in the final standardization phase, reaching the 34th draft version [2] and making it stable and usable for real deployments. The main improvements from version 2 include more efficient header compression, advanced security features based on TLS 1.3, and, especially, the use of QUIC at the transport layer. The resulting protocol stack is thus heavily modified, as we show in Figure 1. QUIC, initially developed by Google, is a transport protocol based on UDP and is currently in the standardization phase too [7]. QUIC revisits TCP, moving congestion control in user space and allowing faster handshakes. Moreover, it solves the long-standing issue of head-of-line blocking, allowing multiple independent streams within the same connection. Indeed, QUIC allows independent retransmission for sub-streams and decouples it from congestion control. This operation is expected to improve users’ QoE with faster website responsiveness, especially in scenarios with poor network conditions. HTTP/3 also mandates the use TLS 1.3 [13], directly incorporated at the QUIC layer. Besides the deprecation of older cipher suites, it allows 1-RTT handshakes and 0-RTT resumption, further reducing session setup time.

Ii-B Related Work

Given its recent conception, few works already targeted HTTP/3. Saif et al. [14] run experiments controlling both client and server accessing a single web page. They study the effect of delay, packet loss and throughput but do not measure any major impact on performance. In contrast to them, we run a large-scale measurement campaign, controlling only the client and targeting thousands of HTTP/3 websites residing on their original servers. Marx et al. [11] compare HTTP/3 implementations, finding a large heterogeneity in how congestion control, prioritization and packetization work. They only run single file downloads, but their results call for extensive in-the-wild measurements. Cloudflare benchmarks its own draft 27 HTTP/3 implementation in [15], finding it to be % slower than HTTP/2. However, their experiments are limited to the blog.cloudflare.com website. Guillen et al. [6] proposed a control algorithm for adaptive streaming tailored for HTTP/3.

Google firstly proposed QUIC in 2012 and, as such, has been the subject of many studies. For example, Wolsing et al. [17] show that QUIC still outperforms TCP thanks to the fast connection setup. Manzoor et al. [10] show how QUIC performs worse than TCP in Wireless Mesh Networks. Wolsing et al. [3] found that QUIC reduces the overall page retrieval time compared with standard HTTP. Kakhi et al. [8] run a large-scale measurement campaign on QUIC, finding that it outperforms TCP in most cases. However, these works target older Google’s QUIC versions, while the current standard proposed at the IETF has made significant progress [9]. Moreover, they focus uniquely on QUIC as the transport layer, neglecting the improvement introduced in HTTP/3 and TLS 1.3, that we aim at measuring in this work.

Iii Data Collection

We rely on two datasets to study (i) the adoption of HTTP/3 and (ii) its performance on diverse network conditions. We summarize them in Table I.

Dataset Visits Goal
HTTPArchive HTTP/3 Adoption
BrowserTime HTTP/3 Performance
TABLE I: Description of the employed datasets.

Iii-a HTTPArchive

We study the adoption of HTTP/3 using the HTTPArchive, an open dataset available online.333https://httparchive.org/,lastvisit:February4th2021 The dataset contains metadata coming from visits to a list of more than M URLs provided by the Chrome User Experience Report.444https://developers.google.com/web/tools/chrome-user-experience-report. The list of URL is built using the navigation data of real Chrome users and contains a representative view of the most popular website and web services accessed worldwide.555HTTPArchive used to adopt the Alexa top-1M website list, but switched to the Chrome User Experience Report when Alexa discontinued the rank in July 2018. Each month, all URLs are visited using the Google Chrome browser from a US data center, and the resulting navigation data is made public. For each visit, the dataset contains information about the page performance and characteristics as well as details on all the HTTP transactions in the HAR format666http://www.softwareishard.com/blog/har-12-spec/, including request and response headers.

Fundamental for our analyses, the details of HTTP responses indicate the eventual Alt-Svc header, which is used by servers to announce support to HTTP/3. By setting the Alt-Svc header, the server has the possibility to inform the client to make subsequent connections using HTTP/3 and may specify the support to specific draft versions (e.g., or ).

We download the HTTPArchive dataset starting from November , when we observe the first websites offering support to HTTP/3. We use the data to study the trend of HTTP/3 adoption. The data sum up to TB, and we store them on a medium-sized Hadoop cluster for scalable processing. Since we are interested in studying the adoption of HTTP/3 on websites, we discard all visits to internal pages (less than half of the totality) and keep only visits to home pages. We refer to this dataset as HTTPArchive.

Iii-B Controlled Experiments

We use the most recent snapshot at the time of writing (December 2020) to build the list of websites currently supporting HTTP/3. We find websites announcing support to HTTP/3. Next, we visit them with all three versions of HTTP (1.1, 2, and 3) to quantify the performance improvements. To this end, we rely on BrowserTime, a dockerized tool to run automatic visits to web pages with a large set of configurable parameters.777https://www.sitespeed.io/documentation/browsertime/ We use BrowserTime to instrument Google Chrome to run visits by using a specific HTTP version. We achieve this using the command line switches of Google Chrome. Important for our goal, Google Chrome allows specifying a set of domains to be contacted with HTTP/3 on the first visit, i.e., without prior indication via Alt-Svc header. We cannot find a similar possibility for Firefox, and, as such, our experiments are limited to Chrome.

Parameter Impairment levels
Latency [ms] Native, 50, 100, 200
Loss [%] Native, 1, 2, 5
Bandwidth [Mbit/s] Native, 1, 2, 5
TABLE II: Impairment levels for controlled experiments.

We are interested in studying the impact of HTTP/3 under different network conditions and, as such, we run our measurements enforcing different network configurations. Each network configuration is defined by three parameters: (i) extra-latency, (ii) packet loss, and (iii) bandwidth limit. To enforce impairments during the visits, we rely on the well-known Linux tc tool. For each parameter, we use different impairment levels, reported in Table II. In case of latency, we impose it on the uplink, while packet loss and bandwidth limit are enforced on both up and downlink. For each network configuration, we visit each website (i) enabling uniquely HTTP/1.1, (ii) with HTTP/1.1 and HTTP/2, and (iii) enabling HTTP/3 too. All visits to the same website are run consecutively and repeated times to get reliable results. Hence, we visit each website times.

BrowserTime collects several statistics for each visit, including details on all HTTP transactions as well as performance metrics. We track two of these that have been shown to be correlated with users’ QoE [4]:

  • onLoad: The time at which the browser fires the onLoad event – i.e., when all elements of the page, including images, style sheets and scripts, have been downloaded and parsed;

  • SpeedIndex: Proposed by Google888https://web.dev/speed-index/, represents the time at which visible portions of the page are displayed. It is computed by capturing the video of the browser screen and tracking the visual progress of the page during rendering.

We run our experiments using two high-end servers connected to the Internet via Gbit/s Ethernet cable and located in our university campus premises. In total, we run visits over a period of one month. The visit metadata account for GB, and we refer to this dataset with BrowserTime.

Iv Results

We first provide an overview of the HTTP/3 adoption offering a historical perspective and analyzing the current status. Then, we study how HTTP/3 affects QoE-related performance metrics by analyzing our experiments under controlled network conditions.

Iv-a Adoption

Fig. 2: Websites announcing support to HTTP/3, separately by IETF draft.
Fig. 3: Server in HTTP response (December 2020).
Fig. 4: Share of objects/volume served on HTTP/3 on enabled websites.

We first study to what extent HTTP/3 has been adopted since its first proposal. To this end, we profit from the HTTPArchivedataset, offering a historical perspective. The first IETF draft is dated January 2017, but we observe the first websites adopting HTTP/3 only in late 2019. Since then, the number of enabled websites started to grow, and Figure 2 shows the trend for the last months of 2019 and the entire 2020. Looking at the Alt-Svc header, we can observe the HTTP/3 draft version supported by the server, shown with different colors in the figure. In case a website offers more than one version, we considered the earliest for the figure. In the first four months, the number of enabled websites increased slowly, reaching 0.7 % of the totality, and we notice that, at that time, only Google and Facebook used to offer HTTP/3 on their websites. In February 2020, the number of enabled websites exploded. This is due to CloudFlare, which enabled HTTP/3 on most of the websites it hosts. The share of enabled websites reaches and passes %, with its maximum in October 2020, with % and  k websites. In November, the number of websites suddenly dropped to less than % ( in absolute terms). We observe that this was caused by CloudFlare suspending support to HTTP/3 due to performance issues, as declared online.999https://community.cloudflare.com/t/community-tip-http-3-with-quic/117551, visited on 2/20/2021. On December, CloudFlare re-enabled HTTP/3 on a subset of websites. On that date, websites were announcing support to HTTP/3, and, in the following, we consider these for our analyses.

The majority of the HTTP/3-enabled websites belong to large companies running their own web server applications. We breakdown this in Figure 3, which indicates the most popular servers as indicated on the HTTP Server header. We extrapolate this figure using again the HTTPArchive dataset, containing details on all HTTP headers of responses. CloudFlare, as expected, hosts most of the HTTP/3-enabled websites (notice the log -scale). Google is in the second position, with GSE (Google Servlet Engine), Google Frontend and GWS (Google Web Server). Indeed, GSE is used on the Blogspot platform, represented by websites in our list. For websites, there is no server indication on HTTP responses, and we find that of these belong to the Facebook company – e.g., facebook.com and instagram.com domains – revealing the social network is a prominent example of an HTTP/3 early adopter. The remaining websites run popular open-source servers (nginx, Apache) or more peculiar HTTP ones (e.g., Caddy), which offer HTTP/3 support in their earlier versions. Interestingly, we do not find any websites whose server header or IP address belong to popular CDNs providers such as Akamai or Amazon CloudFront, meaning they have not started deploying HTTP/3 yet.

Next, we study to what extent objects of enabled websites are served using HTTP/3. Indeed, if a website supports HTTP/3, it does not entail that all objects are sent through HTTP/3. Objects may lay on external CDNs, cloud providers or third parties not supporting the same protocol. This is the case, for example, of ads and trackers typically hosted on different third-party infrastructures. We now use the BrowserTimedataset, which allows us to observe the protocol used for delivering each object composing the web page we visit. Again, we focus on the HTTP/3-enabled websites. In Figure 4, we consider all visits run with HTTP/3 enabled, and, for each, we compute the share of objects served over HTTP/3. As each website is accessed multiple times, we average the values across visits. Clearly, at least the main HTML document is always sent with HTTP/3, but the remaining objects of the page may not. The figure shows the distribution of the percentage of objects served over HTTP/3 (solid red line) and also depicts the byte-wise percentage – i.e., weighting each object by its size. We first notice that in % of cases, all objects are delivered over HTTP/3, meaning that the web page only contains elements hosted on HTTP/3 enabled servers. The websites having % or more of objects (volume) on HTTP/3 are () % and only () % have less than %. Interestingly, we notice that % of websites still have one or more objects retrieved using HTTP/1.1.

Fig. 5: Share of objects/volume served on HTTP/3, separately by provider.

Next, we dissect the above analysis by provider – i.e., the company/CDN hosting the website. We obtain it by looking at the server HTTP header, website name and server IP address, which allow easy identification. As discussed for Figure 3, we notice that HTTP/3 has been adopted mostly by (i) Cloudflare CDN, (ii) the Facebook and (iii) Google companies. The remaining websites (i.e., Other) belong mostly to self-hosted websites running updated versions of the nginx web server. Figure 5 shows the share of objects and volume served over HTTP/3, separately by provider. Cloudflare websites tend to be more heterogeneous, with half of the objects retrieved via non-HTTP/3 servers in median. Moreover, only 24% of the websites’ volume is served by using HTTP/3. This is likely due to the provider’s nature; indeed, Cloudflare offers its hosting service to a very large plethora of websites. These websites may use complex web pages composed of several third-party objects stored on external servers not relying on HTTP/3 yet. Conversely, Facebook and Google show a very different situation. Almost all the websites’ objects and volumes are served with HTTP/3. Indeed Facebook and Google use their CDNs mostly to offer their own services. Looking at Google, the long tail of the distribution is due to Blogspot websites in which the creator may add content from external sources. Finally, considering the Other category, almost all the objects and volumes are served by using HTTP/3. These websites tend to be simple, composed of a few objects stored in the same self-hosted servers together with the main HTML document.

Iv-B Performance

We now study the impact of HTTP/3 on web page performance. To this end, we use the BrowserTime dataset, in which the have been visited multiple times under different emulated network conditions. Using tc-netem, we enforce extra latency, packet loss and limit bandwidth. We then contrast page QoE-related performance indicators (onLoad and SpeedIndex), (i) showing their absolute value and (ii) computing the speed-up. Given a website and a given network scenario, we obtain the speed-up as the relative deviation of the metric when using HTTP/3 () instead of HTTP/2 (). As we always run 5 visits for each case, we consider the median value. As such, the speed-up for a website in scenario is computed as follows:

By definition, is bound in and is negative when a website loads faster under HTTP/3, positive otherwise. We compute it for both onLoad and SpeedIndex.

Fig. 6: onLoad (top) and SpeedIndex (bottom) with extra latency, separately for HTTP/1.1, 2 and 3.

We illustrate how the metric values vary when imposing different network impairments, focusing firstly on extra latency in Figure 6

. Using boxplots, we show the distribution of onLoad (top) and SpeedIndex (bottom), separately by HTTP version (colored boxes). The boxes span from the first to the third quartile, while whiskers report the 10

and the 90 percentiles; black strokes represent the median. When no extra latency is imposed (native case), we observe that onLoad time is in median around s, while SpeedIndex around s, without significant differences across HTTP versions. When adding extra latency, the websites load slower as more time is needed to download the page objects, requiring in median 6 seconds with ms of additional latency. Not shown here for brevity, also packet loss and limited bandwidth cause similar degradation of performance indicators. In the following, we compare to what extent HTTP/3 provides benefits in such scenarios. Figure 6 shows that HTTP/1.1 has the worst performance with high latency, while HTTP/3 shows the greatest benefits. Considering additional latency of ms, websites onLoad in median in , and s with HTTP version 1.1, 2 and 3, respectively.

(a) Latency.
(b) Bandwidth.
(c) Packet loss.
Fig. 7: Speed-up of HTTP/3 on different scenarios. onLoad (top) and SpeedIndex (bottom).

To better catch differences between HTTP/3 and 2, we now study the speed-up in Figure 7, where we show the distribution over the websites for both onLoad (top row) and SpeedIndex (bottom row). The three columns refer to scenarios with additional latency, limited bandwidth and packet loss, respectively. The solid red lines represent the native case – i.e., without imposing any network impairment. Dashed lines represent scenarios with network impairments, as indicated in Table II.

Starting from latency, we confirm what already emerged from Figure 6. When latency is high, HTTP/3 gives sizable benefits compared to HTTP/2. In the native case, we observe no benefit. Looking at the solid red lines, we notice that approximately in 50 % of cases websites load faster with HTTP/3 and in the remaining cases slower. If we impose extra latency of ms, 69 (74) % of websites have lower onLoad time (SpeedIndex), meaning that they load faster. They increase to 76 (81) % with ms latency. With ms, they reach 81 (87) %, and the median speed-up is 0.08 (0.12). In a few words, HTTP/3 shows sizable benefits in case of high latency.

Focusing on experiments with bandwidth limitation (central plots in Figure 7), different considerations hold. We observe sizable benefits only for onLoad time with the bandwith limited to just Mbit/s, where % of websites load faster with HTTP/3. Notice that this benefit cannot be introduced by indirect higher latency due to queuing delay (also called bufferbloat), as we limit the machines’ queue to as little as 32 KB. In other cases, no clear trend emerges, but we only notice a larger variability of the speed-up measure introduced by the constrained setup. For example, in case of SpeedIndex, , , % websites load faster with HTTP/3 with , and Mbit/s bandwidth, respectively. Similar considerations hold for packet loss (right-most plots in Figure 7). Despite a larger variability, we cannot identify any obvious trend, and the speed-up values are equally distributed above and below . In summary, we do not testify any performance benefit of HTTP/3 in scenarios with high packet loss. We only observe improvements with very poor bandwidth and limited to onLoad.

Iv-C Dissecting performance speed-up

We now focus on the performance speed-up as defined previously and discuss how it is impacted by (i) the provider hosting the website and (ii) characteristics of the web pages and load process. As we observed sizable performance benefit for HTTP/3 only in cases of high latency or (very) poor bandwidth, we restrict our analyses to those cases.

Iv-C1 Performance by Provider

Figure 8 shows the distribution of performance speed-up for onLoad, separately by provider. We focus on scenarios with ms extra latency and Mbit/s bandwidth limit. Similarly to Figure 5, we study Cloudflare, Facebook and Google, being the companies behind the majority of the HTTP/3 enabled websites currently. We observe the speed-up largely differs by provider. Focusing on latency (Figure 7(a)), Facebook websites show the highest performance gain ( in median), represented in the figure with the blue dashed line. Moreover, % of websites are loaded faster with HTTP/3 than with HTTP/2. Cloudflare (red solid line) shows the smallest benefits, with only % of websites loading faster. Google and the remaining websites sit in the middle. Similar considerations hold for SpeedIndex, not shown here for brevity.

With limited bandwidth (Figure 7(b)), we observe a very different situation. Here, Facebook has in general worst performance with HTTP/3 with % of websites loading faster with HTTP/2. Conversely, Google (green dashed line) shows the best improvements, with median speed-up and % of websites loading faster with HTTP/3. Cloudflare and the remaining websites exhibit no clear trend with roughly half of the websites loading faster with HTTP/3. Considering the scenario with artificial packet loss, we do not observe differences across providers, as they all approximately behave as shown in Figure 6(c).

(a) Latency.
(b) Bandwidth.
Fig. 8: onLoad speed-up by website provider for scenarios with extra-latency and bandwidth limit.
(a) Latency.
(b) Bandwidth.
Fig. 9: Visit characteristics vs. HTTP/3 speed-up class (normalized values).

Iv-C2 The impact of page characteristics

Now we focus on how web page characteristics impact performance. To this end, we compute various metrics describing the web page load process and contrast them to understand they relationship to the HTTP/3 speed-up. For each visit to the websites in our dataset, in addition to the HTTP/3 speed-up, we compute the following metrics:

  • Number of connections issued by the browser to load the website when using HTTP/3.

  • Number of domains contacted while loading the web page, thus including third-pary domains).

  • Share of objects on the largest connection: to measure to what extent the objects composing the web page are spit into many domains, we compute the share of objects carried over the connection where most objects have been requested, divided by the total number of objects. Remind that the best practices of HTTP/3 recommend avoiding domain sharding to increase performance.

  • Share of objects served on HTTP/3: our goal is to understand how the fraction of the website objects served on HTTP/3 (shown in Figure 4) has an impact on the speed-up.

  • Page Size: to breakdown performance for small and large web pages.

In Figure 9, we compare the distribution of the aforementioned metrics, grouping websites based on classes defined by the onLoad speed-up:

  • H3 Faster: websites loading faster with HTTP/3, i.e., onLoad speed-up .

  • H3 H2: websites having a similar loading time in HTTP/2 and HTTP/3, i.e., onLoad speed-up .

  • H2 Faster: websites loading faster with HTTP/2, i.e., onLoad speed-up .

In the figure, boxes with different colors represent these three classes. The

-axis represents the metrics normalized by scaling to unit variance to ease the visualization and the comparison. Again, we study the scenarios with extra latency (top row) and limited bandwidth (bottom row) since they provide the most interesting insights. In case of extra latency, H3 Faster websites are

%, H2 Faster % and H3 H2 are %. With limited bandwidth, they are %, % and %, respectively.

We first focus on the left-most box group of Figure 9, showing the (normalized) number of connections the browser issued to load the web page. Green boxes hint that websites issuing fewer connections (smaller metric values) are faster with HTTP/3 than in HTTP/2. This true in both scenario, low latency and poor bandwidth. Similar considerations hold if we focus on the second box group, representing the number of contacted domains. Indeed, we notice that the number of connections per website and the number of contacted domains are -correlated. The third box group offers a similar perspective, measuring how web page objects are split on multiple connections/domains. The websites benefiting the most from HTTP/3 are those which tend to mass objects on a single connection – see the highest position of the green boxes, meaning more objects are on a single connection. This is very clear with limited bandwidth (Figure 8(b)), rather than with high latency (Figure 8(a)).

Serving most objects with HTTP/3 (rather than with HTTP/2) has a positive impact too, as we notice from the fourth box group in Figure 9. Again, this is evident especially with bandwidth limit (Figure 8(b)), while with extra latency (Figure 8(a)) it is hard to find a clear trend. Finally, interesting is the case of the web page size (last box group). In scenarios with high latency, the websites benefiting from HTTP/3 are small, while large ones typically perform better with HTTP/2. When bandwidth is scarce, we observe an opposite picture: even if moderately, website loading faster with HTTP/3 are the large ones.

In summary, websites taking benefits from HTTP/3 are those limiting the number of connections and third-party domains, and fully adopting HTTP/3 on all web page objects. Page size has diverse implications depending on the network conditions. These considerations hold in scenarios with high latency or limited bandwidth, while we do not observe any clear trend in case of optimal network conditions or high packet loss, where metric distributions mostly overlap.

V Measurement limitations and future work

We dissect the performance of HTTP/3 under diverse network conditions, showing the impact of network latency and bandwidth and differences across website providers. However, we run measurements uniquely using Google Chrome, as, to the best of our knowledge, it is the only browser that can be instrumented to use HTTP/3 on the first connection – i.e., without the need to observe the Alt-Svc header previously. Moreover, we uniquely visit websites with a fresh browser profile with an empty cache and no pre-existing connections. This limits the scope of our study as we cannot measure how HTTP/3 affects performance on subsequent visits or with a warm HTTP cache.

We measure a large set of websites, but we are limited only to the subset of those already adopting HTTP/3. Moreover, we include only a fraction of those hosted on the CloudFlare CDN, as its HTTP/3 support is partially disabled at the time of writing this article. Our measurements need to be run continuously to thoroughly study how the web ecosystem will react to HTTP/3 in the near future, adopting the protocol and modifying the web page structure to follow the new guidelines, in particular concerning domain sharding.

Finally, HTTP/3 and QUIC are not yet definitive IETF standards. Although recent modifications to the IETF draft only concerned minor protocol features, it will be fundamental to provide a similar analysis once the final standard versions are approved. Similarly, we did not explore how different endpoint configurations affect how objects are sent, eventually retransmitted and congestion control algorithms operate. This calls for fine-grained measurements, better suited to be run on fully-controlled lab experiments. Moreover, we only explore scenarios with uniform and random packet loss, while it is of certain interest to study other loss schemes (e.g., bursty loss).

Vi Conclusions

In this paper, we provided a first study on HTTP/3, measuring its adoption and dissecting performance benefits. We testified how some of the Internet leading companies started deploying it during 2020, although most of the adopter website still host the majority of objects on HTTP/2 third-party servers. With a large-scale measurement campaign, we studied the performance of HTTP/3 under different network conditions, targeting thousands of websites. We found performance benefits emerging in scenarios with high latency or poor bandwidth, while in case of high packet loss HTTP/3 and 2 perform roughly the same. Moreover, we found large performance diversity depending on the infrastructure hosting the website. In general, we observed that websites taking benefits from HTTP/3 are those loading objects from a limited set of third-party domains, thus limiting the number of issued connections.

Acknowledgements

This work has been supported by the EU H2020 research and innovation programme under grant agreement No. 644399 (MONROE) and by the SmartData@PoliTO center on Big Data and Data Science.

References

  • [1] M. Belshe, R. Peon, and M. Thomson (2015-05) Hypertext Transfer Protocol Version 2 (HTTP/2). Request for Comments, RFC Editor. Note: RFC 7540 External Links: Document, Link Cited by: §I.
  • [2] M. Bishop (2021-02-02) Hypertext Transfer Protocol Version 3 (HTTP/3). Internet-Draft Technical Report draft-ietf-quic-http-33, Internet Engineering Task Force, Internet Engineering Task Force. Note: Work in Progress External Links: Link Cited by: §I, §II-A.
  • [3] G. Carlucci, L. De Cicco, and S. Mascolo (2015) HTTP over udp: an experimental investigation of quic. In Proceedings of the 30th Annual ACM Symposium on Applied Computing, SAC ’15, New York, NY, USA, pp. 609–614. External Links: ISBN 9781450331968, Link, Document Cited by: §II-B.
  • [4] D. N. da Hora, A. S. Asrese, V. Christophides, R. Teixeira, and D. Rossi (2018) Narrowing the gap between qos metrics and web qoe using above-the-fold metrics. In International Conference on Passive and Active Network Measurement, pp. 31–43. Cited by: §III-B.
  • [5] R. T. Fielding, H. Nielsen, J. Mogul, J. Gettys, and T. Berners-Lee (1997-01) Hypertext Transfer Protocol – HTTP/1.1. Request for Comments, RFC Editor. Note: RFC 2068 External Links: Document, Link Cited by: §I.
  • [6] L. Guillen, S. Izumi, T. Abe, and T. Suganuma (2019) Sand/3: sdn-assisted novel qoe control method for dynamic adaptive streaming over http/3. Electronics 8 (8), pp. 864. Cited by: §II-B.
  • [7] J. Iyengar and M. Thomson (2020-12-13) QUIC: A UDP-Based Multiplexed and Secure Transport. Internet-Draft Technical Report draft-ietf-quic-transport-33, Internet Engineering Task Force, Internet Engineering Task Force. Note: Work in Progress External Links: Link Cited by: §I, §II-A.
  • [8] A. M. Kakhki, S. Jero, D. Choffnes, C. Nita-Rotaru, and A. Mislove (2017) Taking a long look at quic: an approach for rigorous evaluation of rapidly evolving transport protocols. In Proceedings of the 2017 Internet Measurement Conference, pp. 290–303. Cited by: §II-B.
  • [9] M. Kosek, T. Shreedhar, and V. Bajpai (2021) Beyond quic v1–a first look at recent transport layer ietf standardization efforts. arXiv preprint arXiv:2102.07527. Cited by: §II-B.
  • [10] J. Manzoor, L. Cerdà-Alabern, R. Sadre, and I. Drago (2020) On the performance of quic over wireless mesh networks. Journal of Network and Systems Management 28 (4), pp. 1872–1901. Cited by: §II-B.
  • [11] R. Marx, J. Herbots, W. Lamotte, and P. Quax (2020) Same standards, different decisions: a study of quic and http/3 implementation diversity. In Proceedings of the Workshop on the Evolution, Performance, and Interoperability of QUIC, pp. 14–20. Cited by: §II-B.
  • [12] M. Rajiullah, A. Lutu, A. S. Khatouni, M. Fida, M. Mellia, A. Brunstrom, O. Alay, S. Alfredsson, and V. Mancuso (2019) Web experience in mobile networks: lessons from two million page visits. In The World Wide Web Conference, pp. 1532–1543. Cited by: §II-A.
  • [13] E. Rescorla (2018-08) The Transport Layer Security (TLS) Protocol Version 1.3. Request for Comments, RFC Editor. Note: RFC 8446 External Links: Document, Link Cited by: §I, §II-A.
  • [14] D. Saif, C. Lung, and A. Matrawy (2020) An early benchmark of quality of experience between http/2 and http/3 using lighthouse. arXiv preprint arXiv:2004.01978. Cited by: §II-B.
  • [15] S. Tellakula (2020-04) Comparing HTTP/3 vs. HTTP/2 Performance. Note: https://blog.cloudflare.com/http-3-vs-http-2/ External Links: Link Cited by: §II-B.
  • [16] M. Varvello, K. Schomp, D. Naylor, J. Blackburn, A. Finamore, and K. Papagiannaki (2016) Is the web http/2 yet?. In International Conference on Passive and Active Network Measurement, pp. 218–232. Cited by: §II-A.
  • [17] K. Wolsing, J. Rüth, K. Wehrle, and O. Hohlfeld (2019) A performance perspective on web optimized protocol stacks: tcp+ tls+ http/2 vs. quic. In Proceedings of the Applied Networking Research Workshop, pp. 1–7. Cited by: §II-B.