A Performance Evaluation Tool for Drone Communications in 4G Cellular Networks

04/30/2019 ∙ by Christian Raffelsberger, et al. ∙ 0

We introduce a measurement tool for performance evaluation of wireless communications with drones over cellular networks. The Android software records various LTE parameters, evaluates the TCP and UDP throughput, and tracks the GPS position. Example measurement results are presented.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 2

page 3

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

The commercial and industrial market for applications enabled by small drones is rapidly growing. Opportunities can be found for surveillance, monitoring, transport, emergency management, agriculture, and many other domains. In terms of wireless connectivity, off-the-shelf drones are usually equipped with Wi-Fi, which is sufficient for applications with low coverage requirements. Beyond the range of Wi-Fi, cellular networks could provide wide area coverage. Such long-range connectivity is especially important for autonomous flights going beyond line of sight (LoS) scenarios. However, today’s cellular networks — including Long Term Evolution Advanced (LTE-A) — are not optimized for aerial mobile devices [1]. Most importantly, the base station antennas are typically tilted downwards to serve users on the ground rather than flying devices. Technical enhancements are needed to optimize cellular connectivity for drones, e.g., for interference management and handovers [2]. Upcoming cellular technology — with advanced beamforming, beamsteering, full dimensional multi-input multi-output (FD-MIMO) — may alleviate shortcomings to provide better aerial connectivity [3, 4].

There are several experimental studies on drone communication with Wi-Fi (see [5, 6, 7, 8]) but still few on cellular-based drone communication (see [2, 1, 9, 10, 11]). To study cellular-connected drones, it would be beneficial to have an evaluation tool for the research community that can be used to analyze the performance in current and forthcoming 3GPP (3rd Generation Partnership Project) releases and to design drone applications accordingly. This paper introduces such an evaluation tool: the cellular drone measurement tool (CDMT). It integrates several features that are important to evaluate cellular-based communication in aerial networks. These include the monitoring of signal strength and cell information, assessing TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) performance, and tracking GPS (Global Positioning System) coordinates. Some results obtained with CDMT are shown in this paper to demonstrate the tool in action, flying an Android smartphone with an AscTec Pelican drone. In our experiment, we see some ping-pong handovers and find that the downlink (DL) throughput is higher on the ground whereas the uplink (UL) throughput is higher in the air. More comprehensive results on handovers and throughput using CDMT are given in [12] and [13].

The remainder of the paper is organized as follows. Section II gives an overview of available LTE measurement tools. Section III describes LTE signal metrics. Section IV introduces CDMT and presents its features. Section V shows some example experiments. Section VI concludes the paper.

(a) Configuration (b) Running experiment (c) Results overview
Fig. 1: CDMT screenshots

Ii Related Tools

Some freely accessible LTE measurement tools with various features are available. However, to the best of our knowledge, none of them integrates all the parameters supported by CDMT. Moreover, they usually focus exclusively on TCP as the transport layer protocol for evaluation. Commercial solutions offering similar parameters as CDMT are expensive and often reside within the operator’s network [14, 15].

The performance and power characteristics of LTE, 3G, and Wi-Fi are analyzed in [16]. The Android tool 4GTest [17] is employed to characterize the LTE network performance. It uses a client-server model with multiple servers deployed at different locations and allows users to switch between network types to capture performance measurements.

A model to characterize the round trip time and throughout over the signal strength is derived based on measurements with a customized measurement tool in [18]. The tool uses a client-server model, which records the UDP performance in the DL. The tool is not publicly available and can track only a few parameters of cellular networks.

The Nemo Handy Handheld Measurement Solution [19] is a commercial Android-based network testing application. It provides many relevant parameters, including connection setup delay, download time, time to connect delay, throughput, reference signal received power (RSRP), reference signal received quality (RSRQ), latitude, longitude, altitude, and information about neighboring cells. This tool was used for LTE measurements with a drone in [10] with the result that the signal strength in that scenario increases from  dBm to  dBm until an altitude of  m but then drops again if the drone flies to higher altitudes. Another measurement campaign with a drone and Nemo was made to analyze LTE signals (RSRP and signal-to-interference ratio) in dense environments up to an altitude of  m [11]. Here, the signal strength increases as the drone clears the height of the buildings due to LoS links with the base stations. However, higher path loss and interference from neighboring cells adversely affects the received signal power.

The study [20] addresses LTE and Wi-Fi and analyzes if transport protocols should select the best network or better use multi-path TCP. The app “Cell vs WiFi” [21] measures and compares the performance of LTE and Wi-Fi on Android devices. The app is currently unavailable.

In conclusion, available tools are either proprietary and not freely available or designed in simplicity, not capturing all important parameters needed to analyze cellular-based drone communication. Thus, we developed a measurement tool that supports both TCP and UDP and captures information about signal strength, used frequency bands, and neighbouring cells.

Iii LTE Signal Metrics

The three metrics related to LTE signal strength available on Android devices are RSRP, RSRQ, and RSSNR (reference signal signal-to-noise ratio) [22]. First, RSRP provides the average power received by the resource element carrying the reference signal in any symbol. Typical values range from  dBm to  dBm. It is primarily used for cell selection and handover. However, a high throughput can be observed even with low RSRP. Second, RSRQ provides additional information on when to perform a handover. It indicates the quality of the reference signal with typical values ranging between  dB (worst) and  dB (best). Third, RSSNR is a measure of the signal-to-noise ratio of the received signal, corresponding to the received signal quality. It is used to assess the impact of interference upon the connection and is an additional parameter to perform handover decisions. Typical values range from  dB (best) to  dB (worst).

The UE sends these parameters to the serving base station (eNodeB) as a measurement report. The eNodeB makes the handover decision based on the received reports [23].

Iv Cellular Drone Measurement Tool

CDMT is a performance evaluation tool (see Fig. 1) developed for the Android platform to be used for aerial devices connected with 4G cellular networks. It is available via

https://www.lakeside-labs.com/cdmt

and can be used for academic research. The CMDT records LTE parameters such as RSRP, RSRQ, RSSNR, cell signal quality (CSQ), serving physical cell identity (PCI), channel quality indicator (CQI), E-UTRA absolute radio frequency channel number (EARFCN), neighboring cell information including PCI, EARFCN, RSRP, and RSRQ. It supports throughput measurements for TCP and UDP for DL and UL, packet delay for UDP, and it logs GPS information (time, latitude, longitude, altitude, number of available satellites, speed, and acceleration). All parameters are logged at a rate of  Hz.

In order to perform throughput and round trip time (RTT) measurements, CDMT uses a client-server model, where the client application is executed on an Android device. On the client side, the server network configuration (e.g., IP address and ports) and some further parameters, depending on the type of measurement (e.g., UDP segment size), have to be configured (Fig. 1(a)). Once these parameters are configured and the server application is running, measurements can be performed and recorded (Fig. 1(b)). Graphical representation of RSRP and throughput measurements are shown once the recording has finished (Fig. 1(c)).

The server application is written in Java. It consists of two modules: the control module starts and stops measurements and reports results back to the client; and the data module sends or receives data via TCP and UDP sockets. The UDP data rate is measured on the client for downlink tests and on the server for uplink tests. In the latter case, the server reports the achieved data rate to the client, which stores it in a log file. The TCP data rate is always calculated on the client.

It is assumed that the server is accessible via a public IP (Internet Protocol) address. Hence, we can always perform tests for TCP uplink, TCP downlink, and UDP uplink. UDP downlink tests may not be available depending on the NAT (Network Address Translation) configuration of the mobile operator. CDMT only provides a simple NAT traversal technique: the client first sends a UDP packet to the server in order to create a mapping at the network address translator. The server then uses the external IP address and port combination of the received packet as endpoint for the UDP downlink. However, this mechanism does not work if the NAT changes the port mapping during the measurement. Since this was the case for our mobile operator, we selected an APN (Access Point Name) setting that does not use carrier NAT but instead a public IP address configuration at the end device.

The TCP throughput is measured by downloading or uploading a random stream of data created at the server or client, respectively. Alternatively, the TCP downlink test can be performed using HTTP (Hypertext Transfer Protocol) via a URL (Uniform Resource Locator) pointing to a file, which is then (repeatedly) downloaded by the client, until the measurement is manually stopped. In this configuration, the custom measurement server is not involved. The UDP throughput is measured by sending data packets of configurable size. The first four bytes contain the sending timestamp (to calculate the transmission delay) and the remaining bytes are chosen randomly. This requires that the system clocks at the server and the Android device are synchronized. The calculated delay is reported back using the TCP control socket.

A test is started and ended manually by the user. Measurement values are recorded every second and stored on the client platform for offline analysis. Most existing tools do not support such recording but only provide averaged values at the end.

V Experimental Study

This section describes an experimental study that showcases the possibilities of the CDMT. Further experiments with CDMT, along with more comprehensive results on throughput and handover rates, are presented in [13] and [12].

Fig. 2: Measurement path and trajectory with the location of serving PCIs

V-1 Setup

A Sony Xperia H8216 phone running Android  is used. It has two quad-core processors with  GB RAM and the Qualcomm Snapdragon 845 chipset supporting LTE carrier aggregation (CA). The phone is mounted on an AscTec Pelican drone, which can carry a payload of  g with a flight time of about  minutes. We evaluate the throughput over LTE-A (3GPP Release 13) with ground and aerial measurements. CA is activated in the DL on four frequency bands (LTE800, LTE1800, LTE2100, and LTE2600); no CA is used in the UL. The maximum antenna output power is  dBm.

For UDP experiments a packet size of  bytes is chosen. TCP downlink tests use the file download mode (i.e., a  GB file is repeatedly downloaded from a public server). All other experiments (i.e., TCP UL, UDP UL and DL) are performed by configuring CDMT to connect to a measurement server, located at our lab, running the aforementioned Java module. Measurements are performed as line tests from a reference point to a distance of  m. The drone is set to fly at an altitude of  m for aerial measurements. Fig. 2 shows the flight path trajectory and the locations of the serving PCIs.

V-2 Throughput

Fig. 3 shows the measured throughput over flight distance. Eight curves are given for the throughput in the DL (eNodeB to UE) and UL using either TCP or UDP for both aerial and ground measurements. The presented curves are a mean of three experimental runs. It can be observed that a higher TCP throughput is achieved on the ground than in the air (for both DL and UL) whereas the reverse is true for the UDP throughput in the UL. In the UDP DL, the throughput values on the ground and in the air are similar.

The behavior of the TCP throughput can be explained by a better RSRP on the ground (see Fig. 4). For UDP, the RSRP on the ground is much worse than in the air. However, a concrete relationship between RSRP and throughput cannot be drawn.

Fig. 3: Average throughput on the ground and  m in the air

V-3 Handovers

In the aerial scenario, in the TCP DL, we observed a ping-pong handover from PCI 130 to 388 and back to 130 once in one run; no handover was observed during the other two runs as the UE was always connected to PCI 130 or 92, respectively. In the ground scenario, we observed several ping-pong handovers: in the three TCP DL runs, changes were from PCI 263 to 56 and back to 263, from 130 to 295 and back to 130, and from 92 to 109 and back to 92. In the TCP UL, handovers were from PCI 263 to 56 and back to 263 (in two runs) and from 92 to 109 and back to 92 (in one run). No handover was observed in the TCP aerial UL; the PCIs selected were 92, 263, and 130 for the three runs.

In the UDP DL, no handover was observed, neither on the ground nor in the air. The UE connected to PCI 263 in the air and to PCI 359 on the ground. In the UDP UL, the UE remained connected to PCI 263 in all three aerial experiments and performed a ping-pong handover from PCI 263 to 56 and back to 263 in all three ground experiments.

Fig. 4: Empirical cumulative distribution of RSRP on the ground and  m in the air

Vi Conclusions and Outlook

The cellular drones measurement tool (CDMT) is targeted for aerial devices connected to 4G cellular networks. It can be used to record LTE performance parameters available on Android platforms, including TCP and UDP throughput, and supports tracking via GPS. We demonstrated its feasibility with example results from a measurement campaign over LTE-A (Release 13). These results show better TCP downlink and uplink throughput on the ground and higher UDP uplink throughput in the air (due to better RSRP). Accompanying and future work includes comprehensive experimental evaluations, e.g., assessing handovers and performance at different flight hights. Furthermore, we plan to adapt CDMT for 5G networks.

References

  • [1] B. V. D. Bergh, A. Chiumento, and S. Pollin, “LTE in the sky: Trading off propagation benefits with interference costs for aerial nodes,” IEEE Communications Magazine, vol. 54, no. 5, pp. 44–50, 2016.
  • [2] X. Lin, V. Yajnanarayana, S. D. Muruganathan, S. Gao, H. Asplund, H.-L. Maattanen, M. Bergstrom, S. Euler, and Y.-P. E. Wang, “The sky is not the limit: LTE for unmanned aerial vehicles,” IEEE Communications Magazine, vol. 56, no. 4, pp. 204–210, 2018.
  • [3] N. Tadayon, G. Kaddoum, and R. Noumeir, “Inflight broadband connectivity using cellular networks,” IEEE Access, vol. 4, pp. 1595–1606, 2016.
  • [4] S. D. Muruganathan, X. Lin, H.-L. Maattanen, Z. Zou, W. A. Hapsari, and S. Yasukawa, “An overview of 3GPP release-15 study on enhanced LTE support for connected drones,” arXiv preprint arXiv:1805.00826, 2018.
  • [5] Y. Gu, M. Zhou, S. Fu, and Y. Wan, “Airborne WiFi networks through directional antennae: An experimental study,” in Proc. IEEE Wireless Communications and Networking Conf. (WCNC), 2015.
  • [6] T. Andre, K. A. Hummel, A. P. Schoellig, E. Yanmaz, M. Asadpour, C. Bettstetter, P. Grippa, H. Hellwagner, S. Sand, and S. Zhang, “Application-driven design of aerial communication networks,” IEEE Communications Magazine, vol. 52, no. 5, pp. 129–137, 2014.
  • [7] E. Yanmaz, R. Kuschnig, and C. Bettstetter, “Channel measurements over 802.11 a-based UAV-to-ground links,” in Proc. IEEE GLOBECOM Workshops, 2011.
  • [8] A. A. Khuwaja, Y. Chen, N. Zhao, M.-S. Alouini, and P. Dobbins, “A survey of channel modeling for UAV communications,” IEEE Communications Surveys & Tutorials, vol. 20, no. 4, pp. 2804–2821, 2018.
  • [9] G. Yang, X. Lin, Y. Li, H. Cui, M. Xu, D. Wu, H. Rydén, and S. Redhwan, “A telecom perspective on the Internet of drones: From LTE-Advanced to 5G,” arXiv preprint arXiv:1803.11048, 2018.
  • [10] L. Sundqvist, “Cellular controlled drone experiment: Evaluation of network requirements,” Master Thesis, Aalto University, Finland, 2015.
  • [11] G. E. Athanasiadou, M. C. Batistatos, D. A. Zarbouti, and G. V. Tsoulos, “LTE ground-to-air field measurements in the context of flying relays,” IEEE Wireless Communications, vol. 26, no. 1, pp. 12–17, 2019.
  • [12] A. Fakhreddine, C. Bettstetter, S. Hayat, R. Muzaffar, and D. Emini, “Handover challenges for cellular-connected drones,” in Proc. ACM Workshop on Micro Aerial Vehicle Networks, Systems, and Applications (DroNet), Seoul, Korea, Jun. 2019, accepted paper.
  • [13] S. Hayat, C. Bettstetter, A. Fakhreddine, R. Muzaffar, and D. Emini, “An experimental evaluation of LTE-A throughput for drones,” in Proc. ACM Workshop on Micro Aerial Vehicle Networks, Systems, and Applications (DroNet), 2019, accepted paper.
  • [14] Accuver, “XCAL,” http://www.accuver.com/, Last accessed Mar. 2019.
  • [15] M. Z. Shafiq, J. Erman, L. Ji, A. X. Liu, J. Pang, and J. Wang, “Understanding the impact of network dynamics on mobile video user engagement,” ACM SIGMETRICS Performance Evaluation Review, vol. 42, no. 1, pp. 367–379, 2014.
  • [16] J. Huang, F. Qian, A. Gerber, Z. M. Mao, S. Sen, and O. Spatscheck, “A close examination of performance and power characteristics of 4G LTE networks,” in Proc. ACM Int. Conf. on Mobile Systems, Applications, and Services (MobiSys), 2012.
  • [17] 4GTest, “MobiPerf,” https://github.com/huangshu91/MobiPerf, Last accessed Mar. 2019.
  • [18] M. B. Albaladejo, D. J. Leith, and P. Manzoni, “Measurement-based modelling of LTE performance in Dublin city,” in Proc. IEEE Int. Symp. on Personal, Indoor, and Mobile Radio Communications (PIMRC), 2016.
  • [19] Keysight Technologies, “Nemo Handy Handheld Measurement Solution,” http://www.keysight.com/, Last accessed Mar. 2019.
  • [20] J. Huang, F. Qian, Y. Guo, Y. Zhou, Q. Xu, Z. M. Mao, S. Sen, and O. Spatscheck, “An in-depth study of LTE: effect of network protocol and application behavior on performance,” ACM SIGCOMM Computer Communication Review, vol. 43, no. 4, pp. 363–374, 2013.
  • [21] NMS lab MIT, “Cell vs WiFi,” http://web.mit.edu/cell-vs-wifi/, Last accessed Mar. 2019.
  • [22] J. Cainey, B. Gill, S. Johnston, J. Robinson, and S. Westwood, “Modelling download throughput of LTE networks,” in Proc. IEEE Conf. on Local Computer Networks (LCN) Workshops, 2014.
  • [23] A. Racz, A. Temesvary, and N. Reider, “Handover performance in 3GPP long term evolution (LTE) systems,” in Proc. IST Mobile and Wireless Communications Summit, 2007.