wifi is nowadays present in many devices and is common in households, offices, public institutions, and transportation. Over more than 20 years, many amendments have been made to the original standard, updating both the phy and mac layers to provide higher bit-rate, robustness, and qos.
As users keep asking for higher data-rates, the current deployments struggle to keep up with the demand. One key enabler for gigabit-class communications is the use of the mmw band, which loosely refers to the portion of the electromagnetic spectrum with frequencies higher than 6 GHz. In this frequency range, the amount of available bandwidth is significantly larger than that of the legacy sub-6 GHz counterpart, allowing unprecedented transfer speeds.
As the research started to mature, the wifi Alliance introduced in 2012 the IEEE 802.11ad amendment , standardizing communication in the 60 GHz ism unlicensed band, offering data-rates up to 6.75 Gbps. As a follow-up, its successor IEEE 802.11ay is planned to be standardized by the end of 2020 , introducing technologies such as mumimo, channel bonding, higher-order modulation, and thus even higher speeds. Such extreme data-rates make it possible to unlock new applications, such as wireless office docking, 8K Ultra High Definition video transfer, wireless ar and vr, mobile front-hauling and offloading, etc. .
On the downside, given the higher carrier frequency, mmw transmission suffers from an increased propagation loss, as well as deeper diffraction shadows, and higher penetration and reflection losses, making communication more difficult and less stable.
On the other hand, these characteristics allow for extreme spatial reuse, e.g., transmissions in different rooms will hardly interfere with each other unlike in legacy wifi. Moreover, the short wavelength makes it possible to use antenna arrays with tens of elements packed in a small area, making it is possible to counteract the increased path loss by focusing the radiated power into directive beams, thus increasing the overall antenna gain. While this further reduces interference even where users share the same area and improves spatial reuse, it also creates the problem of directional deafness, worsens the hidden node problem, and makes mobility more complex to handle.
To meet the strict qos requirements of some new applications and partially alleviate the hidden node problem, the standard provides the possibility to transmit data in reserved contention-free periods, that coexist with contention-based access periods, very similar to the legacy csmaca channel access mechanism, and the hybrid allocation can be flexible enough to support the coexistence of traffic with vastly different qos requirements.
In this paper, we present some of the challenges related to the scheduling of IEEE 802.11ad/ay devices in realistic scenarios, with the main focus on the already-standardized IEEE 802.11ad. Furthermore, we discuss some pre-existing works and propose some research directions.
Ii IEEE 802.11ad Overview
Being a mmw-based standard, directional communication with all the added overhead and the possibility of spatially multiplexing users are included in the amendment. To simplify beam management, both the pcpap and the sta divide their surrounding space into sectors as shown in Fig. 1. sta and pcpap will then need to keep beam alignment, which increases the signaling overhead.
Fig. 2 shows that in IEEE 802.11ad time is divided in bi of about 100 ms. Each bi is further divided into bhi and dti, briefly described in the following sections.
Ii-a Beacon Header Interval
The pcpap does most of the managing, such as beaconing, beamforming training, and scheduling, during the bhi. This period can last hundreds of microseconds up to a few milliseconds, and is further divided into three subintervals: bti, abft, and ati.
The bti is used to send dmg Beacons to announce the network, give the basic synchronization and bi structure information, start the beamforming training with new stations, and, if needed, do some basic traffic management. Beacons are sent over the different sectors, covering all possible directions to maximize coverage for untrained sta.
After receiving a dmg Beacon during the bti, new sta can use the abft to complete the basic beamforming training by sending ssw frames in different sectors. Beam alignment is completed once the pcpap responds with an ssw Feedback.
Finally, advanced scheduling mechanisms setup and further network management can be done during the optional ati.
Ii-B Data Transmission Interval
The dti is mainly used for the actual data transmission, but it can also be used to improve communication links and for further scheduling. The dti comprises cbap and sp, which can appear in arbitrary combinations and are scheduled during the bhi.
Transmission in cbap follows the rule of edca, slightly modified to account for directional transmission, in which sta compete with each other in order to transmit their data.
Instead, sp are scheduled contention-free intervals that are dedicated to exclusive transmission between a pair of sta111A pcpap also contains a sta, i.e., a logical entity that is a singly addressable instance of a mac and phy interface to the wireless medium . to guarantee qos. The standard also allows for spatial sharing, meaning that multiple pairs of sta with low cross-interference can be scheduled in the same sp. This, however, comes at the cost of increased overhead since interference measurements must periodically take place.
Iii Scheduling in IEEE 802.11ad
IEEE 802.11ad allows for great flexibility in the scheduling of radio resources, but we will hereby describe only some of these possibilities in their simplest form.
We want to stress the fact that, unlike in traditional contention-based medium access, scheduled sp guarantee qos. ac introduced in 802.11e, in fact, only allow for stochastic traffic prioritization according to the DiffServ paradigm, which ceases to work in congested networks. For this reason, allocated traffic is especially important for those applications with strict qos constraints. Instead, more realistic applications, such as data transfer or asynchronous bursty traffic, can simply rely on cbap.
As shown in Fig. 3, a sta can set up an allocation by sending an addts Request frame to the pcpap during the dti and embedding a dmg tspec element. The dmg tspec element is created by the requesting sta and comprises information such as the allocation period, and the minimum and maximum allocation duration.
Based on its admission policy, the pcpap will either reject or accept the request, immediately notifying the requesting sta via an addts Response. If accepted, the allocation is made effective by including it in the ese transmitted in the next dmg Beacons, which will contain details such as the effectively allocated period duration and the sp start time. In this way, sta not involved in the communication will not create interference and will be able to switch to power-saving mode. Otherwise, the pcpap can either reject or propose a change in the dmg tspec. A sta can later update the dmg tspec by sending another addts Request with the updated element and follow again the same procedure.
Allocating the right duration to sp is clearly a trade-off between qos traffic, which needs resources to fulfill the minimum requirements imposed by the application, and elastic traffic, which still needs resources even though with less stringent requirements. Since resource availability, as well as channel quality, are time-varying, the standard supports sp extension and truncation services, which let the stations keep transmitting and/or relinquishing the unused occupied channel. Still, these features bring extra overhead and should thus be used carefully.
A mathematical model for preliminary allocation of sp for vbr flows is presented in , which helps determine how to set the tspec parameters to meet qos requirements while minimizing the amount of allocated time. Unfortunately, sp are assumed to be placed at the beginning of the dti, which is not possible in general for applications with tight delay constraints. For example, for virtual or augmented reality services, latencies should be below 20 ms to avoid motion sickness.
Other works in the literature consider different aspects of the dti. For example,  derives the theoretical maximum throughput for cbap when two-level MAC frame aggregation is used. Beamforming is also considered in , which proposes a joint optimization of beamwidth selection and scheduling to maximize the effective network throughput, while other works, though not specifically concerning IEEE 802.11ad, deal with transmission scheduling for mmw communications .
Iv Future Research
In this section, we highlight some possible research directions. In particular, in Sec. IV-A we describe the main tools that are currently available to study the subject. Then, in Sec. IV-B we propose a possible research plan.
Iv-a Available Research Tools
Although commercial devices supporting the IEEE 802.11ad standard are currently available, manufacturers do not provide tools to access low-level functionalities. Ultimately, it is more flexible, timely, and cost-effective, although arguably less realistic, to simulate the behavior of such devices.
In particular, significant effort has already been done implementing the IEEE 802.11ad standard into ns3 
, a popular open-source network-level simulator. The last release of the simulator also supports quasi-deterministic channel modeling based on ray-tracing, making simulations extremely accurate and realistic at the cost of a long preliminary channel generation phase, although some works already tried to improve this aspect. While the implementation already covers most parts of the standard, it is still missing the scheduling mechanisms necessary for this project. The authors of  are also working on the implementation of the IEEE 802.11ay amendment , making their work even more valuable.
Historically, scheduling algorithms have been mainly based on heuristics, trying to balance performance and adaptiveness versus complexity. In the last years, instead, the ml revolution has brought many innovations also to the telecommunication field at all layers of the stack and, in particular, rl is especially applicable to optimize or even replace legacy scheduling algorithms . Following the principle of self-driving networks , ml algorithms can learn from real on-line data and supersede manually-designed protocols, which are becoming increasingly complex. OpenAI Gym is one of the most used rl toolkits and has been adopted by all popular ml frameworks. Given their potential in many fields of networking and telecommunications in general, OpenAI Gym APIs have also been integrated into ns3  with the name of ns3-gym.
With these powerful tools, it will be possible to further advance the state of the art, create a comprehensive performance evaluation of available algorithms and further improve upon them once the weak points are clearly identified.
Iv-B Research Plan
One of our first goals is to extend the already existing IEEE 802.11ad ns3 module with the necessary mechanisms to make it properly support the hybrid channel access and advanced scheduling (see Sec. IV-A), and add the support to the ns3-gym framework. A significant development effort will be put into the creation of a proper simulation environment, with particular attention to the computational complexity since a high data-rate simulation of just 10 s of simulated time may currently take one hour or more of run-time. This makes the design, evaluation, and optimization of scheduling protocol a lengthy process, which may be even infeasible if rl is involved since many training episodes are needed to learn even basic mechanisms.
Indeed, decisions such as admission policy, resource allocation, smart sp truncation or extension, and spatial sharing are often difficult to accurately model in terms of trade-offs and usually comprise several tunable parameters. However, if trained correctly, an rl agent is often capable of learning extremely complex rules and optimize the network for different networking metrics (e.g., delay, jitter, throughput, fairness) even beyond complicated heuristics.
Resource allocation can be divided into two subproblems. Specifically, sta have to translate information given by the application into dmg tspec elements and the pcpap subsequently has to efficiently schedule the dti especially considering the mcs used. Regarding the former, applications may not yield constant inter-packet arrival time (e.g., frame-rate drop in video applications) nor packet size (e.g., when compression is considered). At the same time, transmission conditions may vary mainly due to environmental changes, mobility, or even blockage, thus increasing performance variability. If qos requirements are not met, the rl agent of a sta could thus update the tspec.
On the other hand, the pcpap has to allocate sp for a bi based on the available resources. Effective scheduling must take into account, in addition to network metrics, the possible evolution of the mcs since the packet transmission time largely depends on it. Given the significant differences in channel dynamics of IEEE 802.11ad with respect to sub-6 GHz wifi, new ones can be proposed to account for the specific characteristics of the mmw channel. An rl agent could thus jointly adapt the mcs and perform scheduling to optimize the network performance by observing the evolution of both channel statistics and network traffic.
One way to overcome the problem of slow simulations is to quickly pre-train the rl agent to make it learn at least simple decisions, such as understanding when a new request does not fit the available resources, avoid overlapping sp during scheduling, and avoid scheduling highly cross-interfering users with spatial sharing. Thus, we plan to build a very simple and fast simulator that will only model relevant notions, e.g., basic channel and traffic modeling and the bi structure defined in IEEE 802.11ad, but eliminating the fine details which make ns3 realistic but extremely slow. In this way, the agent can learn very broadly which actions it should take and then fine-tune its behavior via more realistic simulations. Then, to further decrease ns3 simulation run-time, a database of simulation results can be created and multiple agents can passively learn from it 
before fine-tuning their performance on ad hoc simulations. Transfer learning will also be considered to speed up convergence to effective policies in different scenarios.
Another objective will be to understand the traffic behavior of target applications. For example, it could be possible to acquire real-world traffic traces of ar/vr applications, characterizing and modeling their traffic patterns with focus on packet size, and variability of inter-packet arrival accounting for variable frame-rate statistics. These models would ultimately be integrated with standardized scenarios [16, 17] to further increase simulation realism.
Furthermore, understanding how the current state of the art performs in a realistic simulator will allow understanding the strong and weak points of the proposals in realistic settings. From detailed studies, it will be possible to understand how the state of the art can be improved upon with heuristics or, when modeling becomes too complex or inaccurate, ml-based approaches.
These results will then be easily transferred to the future IEEE 802.11ay standard, which will add further complexity on top of the already existing one, by introducing channel bonding and mumimo. Even more complex schedulers will then have to be designed, but starting from the solid ground of the proposed work further improvements will be possible.
In this paper we briefly described the main characteristics of IEEE 802.11ad, mainly focusing on the mac layer and especially on the newly introduced scheduling mechanisms, allowing different types of traffic to coexist and potentially improving the performance of qos-sensitive applications. As shown in Sec. III, some research has already been done in this direction but lacks a common and realistic testing ground, making it unclear whether the assumptions may hold.
Our future work will focus on proposing solutions for the many open problems described in Sec. IV-B. Models and source code that will be considered of interest for the community will be published, making it possible to fairly compare results from different groups in a common and realistic simulation environment.
-  IEEE Standard for Information technology—Telecommunications and information exchange between systems Local and metropolitan area networks—Specific requirements - Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications, Std., Rev. IEEE Std 802.11-2012, Dec. 2016.
-  “Status of project IEEE 802.11ay,” Task Group ay, available: http://www.ieee802.org/11/Reports/tgay_update.htm.
-  Task Group ay, “IEEE 802.11 TGay use cases,” Aug. 2017, doc.: IEEE 802.11-2015/0625r7. [Online]. Available: https://mentor.ieee.org/802.11/dcn/15/11-15-0625-07-00ay-ieee-802-11-tgay-usage-scenarios.pptx
-  T. Nitsche, C. Cordeiro, A. B. Flores, E. W. Knightly, E. Perahia, and J. C. Widmer, “IEEE 802.11ad: directional 60 GHz communication for multi-gigabit-per-second Wi-Fi,” IEEE Communications Magazine, vol. 52, no. 12, pp. 132–141, Dec. 2014.
-  E. Khorov, A. Ivanov, A. Lyakhov, and V. Zankin, “Mathematical model for scheduling in IEEE 802.11ad networks,” in IFIP Wireless and Mobile Networking Conference (WMNC), Jul. 2016, pp. 153–160.
-  M. U. Rajan and A. Babu, “Theoretical maximum throughput of IEEE 802.11 ad millimeter wave wireless LAN in the contention based access period: with two level aggregation,” in International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET). IEEE, Mar. 2017, pp. 2531–2536.
-  H. Shokri-Ghadikolaei, L. Gkatzikis, and C. Fischione, “Beam-searching and transmission scheduling in millimeter wave communications,” in IEEE International Conference on Communications (ICC). IEEE, Jun. 2015, pp. 1292–1297.
-  W. Feng, Y. Wang, D. Lin, N. Ge, J. Lu, and S. Li, “When mmWave communications meet network densification: a scalable interference coordination perspective,” IEEE Journal on Selected Areas in Communications, vol. 35, no. 7, pp. 1459–1471, Jul. 2017.
-  H. Assasa, J. Widmer, T. Ropitault, and N. Golmie, “Enhancing the ns-3 IEEE 802.11ad model fidelity: beam codebooks, multi-antenna beamforming training, and quasi-deterministic mmwave channel,” in Workshop on Ns-3 (WNS3), Jun. 2019, pp. 33–40.
-  M. Lecci, P. Testolina, M. Giordani, M. Polese, T. Ropitault, C. Gentile, N. Varshney, A. Bodi, and M. Zorzi, “Simplified ray tracing for the millimeter wave channel: a performance evaluation,” in Information Theory and Applications Workshop (ITA), Feb. 2020.
-  H. Assasa, J. Widmer, J. Wang, T. Ropitault, and N. Golmie, “An implementation proposal for IEEE 802.11ay SU/MU-MIMO communication in ns-3,” in Workshop on Next-Generation Wireless with Ns-3 (WNGW), 2019, pp. 26–29.
-  T. Azzino, T. Ropitault, and M. Zorzi, “Scheduling the data transmission interval in IEEE 802.11 ad: A reinforcement learning approach,” in International Conference on Computing, Networking and Communications (ICNC). IEEE, Feb. 2020, pp. 602–607.
-  N. Feamster and J. Rexford, “Why (and how) networks should run themselves,” in Applied Networking Research Workshop (ANRW). ACM, Jul. 2018, pp. 20–20.
P. Gawłowicz and A. Zubow, “Ns-3 meets OpenAI gym: The playground for machine learning in networking research,” inACM International Conference on Modeling, Analysis and Simulation of Wireless and Mobile Systems (MSWiM), Nov. 2019, pp. 113–120.
-  R. Munos, T. Stepleton, A. Harutyunyan, and M. G. Bellemare, “Safe and efficient off-policy reinforcement learning,” in International Conference on Neural Information Processing Systems (NIPS), Dec. 2016, pp. 1054–1062.
-  Task Group ad, “TGad evaluation methodology,” Jan. 2010, doc.: IEEE 802.11-09/0296r16. [Online]. Available: https://mentor.ieee.org/802.11/dcn/09/11-09-0296-16-00ad-evaluation-methodology.doc
-  Task Group ay, “TGay evaluation methodology,” Jan. 2016, doc.: 802.11-15/0866r2. [Online]. Available: https://mentor.ieee.org/802.11/dcn/15/11-15-0866-04-00ay-11ay-evaluation-methodology.doc