A Primer on Large Intelligent Surface (LIS) for Wireless Sensing in an Industrial Setting

06/11/2020 ∙ by Cristian J. Vaca-Rubio, et al. ∙ NTNU Aalborg University 2

One of the beyond-5G developments that is often highlighted is the integration of wireless communication and radio sensing. This paper addresses the potential of communication-sensing integration of Large Intelligent Surfaces (LIS) in an exemplary Industry 4.0 scenario. Besides the potential for high throughput and efficient multiplexing of wireless links, a LIS can offer a high-resolution rendering of the propagation environment. This is because, in an indoor setting, it can be placed in proximity to the sensed phenomena, while the high resolution is offered by densely spaced tiny antennas deployed over a large area. By treating a LIS as a radio image of the environment, we develop sensing techniques that leverage the tools of image processing and computer vision combined with machine learning. We test these methods for a scenario where we need to detect whether an industrial robot deviates from a predefined route. The results show that the LIS-based sensing offers high precision and has a high application potential in indoor industrial environments.



There are no comments yet.


page 3

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Massive multiple-input-multiple-output (MIMO) is a primordial technology in the 5th generation of wireless networks (5G) with the main purpose of increasing area spectral efficiency [8]. In massive MIMO, the base station is equipped with a very large number of antennas. Looking towards post-5G, researchers are defining a new generation of base stations that are equipped with an even larger number of antennas. The concept of Large Intelligent Surface (LIS) designates a large continuous electromagnetic surface able to transmit and receive radio waves. These large surfaces are placed on walls for example and are easily integrable into the surroundings. In practice, a LIS is composed of a collection of closely spaced tiny antenna elements. While the potential for communications of LIS is being investigated, these devices offer possibilities which are not being under study accurately, for instance, the environment sensing. Indeed, such large surfaces contain many antennas that can be used as sensors of the environment based on the Channel State Information (CSI). The first purpose of this work is to provide an overview of sensing techniques in the context of LIS by exploiting the information provided by the radio propagation environment for 6G.

On a related note, machine learning based approaches are providing suitable solutions to the optimization problems in the area of the massive MIMO systems [7]

. Due to the large dimensions of the system in extra-large arrays it is crucial to use deep learning to exploit complex patterns of information dependency between the transmitted signals. The input to this deep learning networks could be addressed from different perspectives: channel state information (CSI), reconstruction of feature maps to exploit convolutional neural networks for instance. The output could be several parameters depending on the optimization problem we are trying to solve. For example, with a radio environmental image, computer vision approaches can be exploited for tracking users and performing any strategy for bitrate optimization. Because the network can be trained offline, one of the key benefits of this method is that we can improve the LIS performance in a real time fashion.

Currently, wireless networks exploit radio waves for communications mostly. However, another advantageous way of using radio signals is for sensing the environment. The main goal of sensing is taking advantage of the radio propagation environment in order to describe what is occurring in space in a specific scenario.

CSI acquisition Robin?

Sensing is a major feature that has been addressed in the literature in different ways. Above all, we could resume several sensing techniques on three different ways which could be categorized as follows:

Fig. 1: Communication signals
Fig. 2: Tomography-like
Fig. 3: Radar-like

I-a Sensing communication signals.

A powerful strategy for sensing is based on the communication signals that are present on a certain location. Real time fall detection systems have been developed [19] using commodity WiFi devices operating on the 5 GHz band. Specifically, abrupt changes in CSI are mapped into fall detection metrics. Those relying on phase differences between two receive antennas are shown to be more sensitive than alternatives based on amplitude information. In [13] the auhors present WiSee, a gesture recognition system that uses WiFi signals to enable whole-home sensing and recognition of human gestures. WiSee works by using the Doppler shifts and multi-path distortions caused by human motion over the WiFi signals in the environment. Since higher transmission frequencies result in a higher Doppler shift for the same motion, WiSee operates at the 5 GHz band for a better resolution. Similar to the aforementioned approaches, a LIS could be useful for sensing the environment. As observed in Figure 3, the LIS would be placed on the bottom wall. Then, signals traveling through the air can impinge into obstacles and their reflection generates a perturbance on the measurement-based wall image obtained in the surface. In this way, thanks to the high resolution image provided by the surface, it could be possible to describe the events taken place in the nearby by providing a radio feature map based on the radio propagation environment.

I-B Radio tomographic imaging.

The tomography image reconstruction is a concept that has been envisioned from 2010 in Wireless Sensor Networks (WSN). Radio Tomographic Imaging (RTI) is an RSS-based technology for rendering physical objects in wireless networks [21]

. In order to do so, one deploys nodes forming a WSN within an area. Each node operates in the 2.4 GHz frequency band, and uses the IEEE 802.15.4 standard for communication. Then every node transmits by taking turns to the rest of the nodes so the receivers measure the Received Signal Strength (RSS) of each link. After an initial calibration, RSS measurements are incorporated on a linear model to reconstruct images of moving objects. This is done by relating them to the change in attenuation occurring in a network area. An investigation of noise statistics in dynamic multipath environments is performed. They also describe an error bound on image estimation for a given node geometry and end up by discussing the ill-posed problem of RTI providing a regularized Tikhonov solution for obtaining an attenuation image. In

[24] an improvement of [21] has been made by basing their study in the variation of the RRS levels within a time window so no calibration is needed. However, if an object remains in the same position exceeding that time, it would disappear from the reconstructed image. In [10] authors propose a variational Bayes approach. Similarly, an alternative LIS-based problem could be formulated by splitting them into sections that are transmitting one after another while the rest are receiving, scanning in this way the environment as represented in Figure 3.

I-C Radar-like/RF sensing.

There are several applications by using the reflection of the wireless communication signals pointed towards a measured target. For example, in [23] Wi-Fi signals have been used for human pose recognition. The authors use Wi-Fi signals due to their capacity to traverse walls and obstacles. The main feature of these signals is that they reflect off the human body. Then, to separate RF reflections from different objects, they use the FMCW (Frequency Modulated Continuous Wave) technique. Thanks to this, they process the RF reflections by a single pair of horizontal and vertical heatmaps so they use a T-shape antenna array for acquiring both. Then, cross-modal supervision following a teacher-student network is used for transferring visual knowledge of human pose using synchronized images and RF signals. In this way, once the model is trained, they are able to estimate human pose by just using Wi-Fi reflections even through walls. In [1], they use a wireless sensing technology for tracking human breathing by using low-power signals and monitoring their reflections off the human body. Similarly to the previous approach, a LIS could be splitted into sections transmitting by turns, although this time a dedicated signal towards the measured target is used, as represented in Figure 3.

Notice that S2 and S3 use dedicated signals so they do not require active devices at the entity to be sensed.

The rest of the paper is organized as follows: in Section II, a description of the modeled use case is presented as well as the concept of holographic image. Then, in Section III a description of the machine learning model and its corresponding evaluation metrics is introduced. Next, in Section IV a discussion is developed around the impact of spacing, aperture and array configuration in the evaluation performance for our presented use case. Finally, some remarkable conclusions are made in Section V.

Fig. 4: Use case scenario
Fig. 5: Holographic image
Fig. 6: Noisy Holographic image with SNR=-60 dB

Ii Problem formulation and model description

In order to simulate the operation of these surfaces in the presence of the multipath components of signals propagation, we are making use of a Ray Tracing software, specifically, ALTAIR FEKO Winprop [5] - [22]. Ray tracing is a strategy which leads to provide very accurate results [14] - [3], although its computational cost increases exponentially with the maximum allowed number of reflections and diffractions [17].

Ii-a Simulated scenario

For the purpose of this work, we conducted simulations with the ray-tracing software. The baseline set-up is described in Figure 6. A typical small size industrial scenario of size 484 . Several robots are placed in a fixed position in the deployment while a red-coloured one follow a horizontal fixed route. This latter robot has a transmitter on the top, which would allow the surface to sense the environment according to the signal reflections within the environment. Then, a LIS is deployed in the bottom wall. The goal is to detect anomalies over the robot’s predefined route using as information the image generated by the received power in each of the antenna elements within the surface. The robot continuously transmit a sensing signal of fixed power. Similarly, the same scenario with the abscense of scatterers in a LOS deployment has been tested to leverage the performance potential of LIS for sensing. Concretely, we have defined the correct route of the robot as a horizontal trajectory, and the anomalous routes as three horizontal trajectories displaced 50/100/150 cm from the correct route. A visual explanation is depicted in Figure 7.

Fig. 7: Correct robot route (blue) vs anomalous routes (orange)

For these routes, we will simulate in the ray tracing software time steps which corresponds to different positions of the robot in both the correct and anomalous routes. Then a holographic image snapshot of the measurements is taken in every .

Table I describes the most relevant parameters used for the simulation

Surface width
dimension (m)
Surface height
dimension (m)
Spacing (cm)
3.5 20 20 Omni 22 8 Free Space
TABLE I: Parameters
Fig. 8: VGG19 feature extractor

Ii-B Received power and noise modeling

We model the received power in every antenna element as the superposition of the electric field strengths (amplitude and phase) of every ray path in every antenna element. In this way, for every antenna element we have to calculate and [11]:


where refers to the number of used rays, to the amplitude of the electric field strength and to the phase for the ray path in a specific antenna element. Being then the total electric field strength:


Taking into account the free space propagation model, the relationship between the electric field and the received power (in dBm scale) is defined as:


being the free space impedance of value .

For testing the algorithm under different scenario conditions, noise has been modelled as a zero-mean Gaussian variable power, which satisfies a target mean SNR () along the route described as:


,where refers to the total amount of antenna elements in the surface, accounts for the total number of time steps simulated in the software and the received power for the antenna element in the time step.

Ii-C Holographic images

The concept of holography is predicated on the principle of interference. A hologram captures the interference sample between two or more beams of coherent light. One beam is shone directly at the recording medium and acts as a regard to the mild scattered from the illuminated scene. The hologram is the recorded interference pattern as a result of constructive and destructive combinations of the superimposed light-wavefronts. By the utilization of a coherent laser light-source and a stable geometry the interference sample is sitting and will be recorded into the hologram’s photosensitive emulsion. The hologram is then chemically processed in order that the emulsion features a modulated density, freezing the interference pattern in this way [18] - [6]. An holographic image in the context of LIS could be described as a structure which uses electromagnetic wireless signals impacting in a determined scatterer in order to obtain a profile of the environment. A LIS is able to sense the environment in a really accurate manner. In these way, these surfaces could be useful for providing a high resolution image of the radio propagation environment based on the received power in every antenna element. As an illustration, a holographic image of the aforementioned scenario would ideally look like Figure 6 in the absence of noise and like Figure 6 in the presence of it. Thanks to the large aperture offered by the surface, we are able to reconstruct a feature map (image) that describes what is occurring in space, based on the information acquired from the radio propagation environment.

Iii Machine learning model

For the purpose of our use case, a machine learning model is used for the classification of the anomaly in the robot’s route to leverage the peformance of the LIS in the industrial scenario. We have a limited space of data points that can be used for our experiment, resulting in a small dataset. Then, being this said, note that one of the main problems to face when using machine learning algorithms is their hunger for data. The best solution to tackle this matter is by making use of transfer learning

[12]. Transfer learning could be defined as taking advantage of utilizing existing networks (which represents knowledge) from the source learner (pre-trained model) in the target task (new classification task). Among the available strategies for transfer learning [12], we use feature representation.

One of the fundamental requirements for transfer learning is the presence of models that perform well on already defined tasks. The majority of state-of-the art deep learning architectures have been openly shared by their respective researchers. These models are usually shared in the form of the millions of parameters/weights the model achieved while being trained to a stable state [15]

. These pre-trained models are available for everyone to use. The famous deep learning Python library, Keras

[4], provides an easy way to reuse some of this popular models. In our case, we are making use of the VGG19 [16] architecture.

The process is explained in Figure 8

. The first row, corresponds to the VGG19 architecture. In order to perform the feature extraction, we remove the last Fully Connected layer (FC) that performs the classification for the purpose of VGG19 and modify it for our specific classification task (anomaly/not anomaly in robot’s route). We note that we have frozen the architecture meaning that the weights and biases in VGG19 are not re-trained along the process. Instead, they are re-used to generate the features that will be later used to feed the Support Vector Machine (SVM) binary classifier. Hence, only the SVM will be trained by using the extracted features.

Iii-a Dataset format

Our machine learning algorithm consists of two parts: (I) feature extractor by using VGG19 and a (II) SVM binary classifier

Then, the dataset used contains samples (holographic image snapshots of received power) which corresponds with the total time steps simulated in the software, being then the total number of samples . The input will be an image represented by a matrix with channels (RGB) of size and pixels. Our data can then be denoted as , where is the input features matrix and is the corresponding desired output label associated to the image (anomaly or not anomaly).

Once the feature extraction is performed the output will be a channels of size and pixels. As for training the SVM vectors are needed, this data is reshaped into an input feature vector formed by features, being now our data , where is the -dimensional training input features vector (being ), is the value of the feature, and is the corresponding desired output label vector (anomaly or not anomaly).

In this way, the dataset used for performance evaluation is composed of 365 simulated time steps, being then

radio propagation snapshots containing images of anomaly and not anomaly situations. The dataset is splitted into a 80% training set, 20% for the test set. During the training, the hyperparameter

is tuned to prevent overfitting by controlling the balance between bias and variance in the SVM model. The optimum value used is

, which was identified by using a 5-fold cross-validation strategy [2].

Iii-B Evaluation metrics

For evaluating the prediction effectiveness we have used the following evaluation metrics common in the ML literature:

  • Precision positive (PP) and negative (PN) as the proportion of correct predictions of a given class

  • Recall positive (RP) and negative (RN) as the proportion of actual occurrences of a given class which has been correctly predicted.

  • Positive F1-Score () and Negative F1-Score(

    ) as the harmonic mean of precision and recall.


Then, for the evaluation of our results we are focusing on and that are useful to take into account both precision and recall. Our positive class stands for anomaly while the negative for no anomaly.

We will use these metrics to evaluate the performance of our approach, analyzing the influence of aperture, spacing, array configuration and noise.

Iv Discussion

Note that in this scenario, it is way more important to avoid undetected anomalies than having a false positive when no anomalies occurred. Then, the most useful part is the evaluation of the anomalous results ( score) and we are therefore centering our discussion mostly around this metric. However, for the sake of clarification, a summary of the results according to the score is provided in case the reader is interested.

Iv-a Impact of spacing and aperture

For the evaluation of the approach, different spacings are considered with respect to the wavelength (, and ).

Fig. 9: Spacing impact in performance

We have extracted several horizontal arrays at the middle height of the wall in order to evaluate the importance of the aperture and spacing for a given SNR scenario (30 dB). The spacing of which is far from the concept of LIS is presenting really innacurate results showing that the resolution obtained for that spacing is not precise enough. Here it is highlighted the effect of densification for a given aperture, as it can be seen the lowest spacing leads to the highest results. In this way, it is seen the potential of the LIS concept regarding closely spaced antenna elements.

Table II shows a summary of the results for both metrics.

Apertures (m)
10.88 5.44 2.72 1.36 0.68
(%) 85 79 71 62 62
(%) 78 70 59 56 48
(%) 80 72 69 60 52
(%) 69 60 58 52 45
(%) 67 61 61 51 49
(%) 55 54 46 41 38
TABLE II: Results Summary aperture vs spacing

Iv-B LIS and horizontal array comparison

To show the potential of LIS for sensing a comparison between using the LIS and several horizontal array configurations taken at middle height of the LIS (128, 64 and 8 antennas) are considered for the best spacing case (). The different array configurations are depicted in Figure 10.

Fig. 10: Different array configurations
Fig. 11: Performance comparison

A comparison along several SNR scenarios are presented in Figure 11. In general, we see that performance is worsened as the noise is increased or the aperture is decreased. Here, the motivation of using LIS for sensing is two folded. First, the LIS system outperforms the different array configurations with a significant difference. Second, the LIS system robustness to noise in the extreme SNR scenario (-60 dB) shows the potential of using a full holographic image which captures accurately the macroscopic pattern of the propagation environment. What is more, it shows that using a sensing signal of really low power is a good approach which could be considered.

The results are summarized in Table III

SNR (dB)
20 0 -20 -60
LIS (%) 95 94 94 90
(%) 93 92 92 84
Array 128 (%) 85 85 83 63
(%) 79 78 77 47
Array 64 (%) 80 79 77 54
(%) 73 70 70 41
Array 8 (%) 67 62 57 49
(%) 53 49 48 48
TABLE III: Results Summary LIS vs array configurations

Iv-C Industry and LOS scenario comparison

As a final comparison, the difference in performance between the industrial scenario presented and a similar one in which the scenario is empty (simulating in this way an ideal LOS scenario with the abscense of scatterers) has been tested.

Fig. 12: Performance comparison

The results could look counterintuitive at a first glance. The reason why LOS scenario is behaving better from 20 to -20 db is because of the parallel trajectories under study. Because of having a deterministic propagation model, the power recieved in every antenna element depends strictly on the pathloss for a determine distance. Due to the fact of being parallel, the classification task resides in discerning between the received power levels. Then, regardless of which points are being classified, the further away trajectory (the correct one) is having a mean less received power along the surface. In this way, the LOS case is going to perform better until the noise power is high enough to hide this power difference between a point of the correct route or a point of the anomalous route. This effect is seen with the noise is really extreme in the -60 dB SNR case, in which the scenario with scatterers clearly outperforms the LOS one.

The results are summarized in Table IV.

SNR (dB)
20 0 -20 -60
LIS scatterer
(%) 95 94 94 90
(%) 93 92 92 84
(%) 99 99 99 66
(%) 98 98 98 63
LIS scatterer
(%) 95 95 95 90
(%) 93 93 93 84
(%) 100 100 100 64
(%) 100 100 100 49
LIS scatterer
(%) 91 91 91 83
(%) 87 87 87 75
(%) 99 99 99 53
(%) 98 98 98 49
TABLE IV: Results Summary LIS scatterer vs LIS LOS

V Conclusions

Large Intelligent Surfaces are a key ingredient in current studies for improving communications in the forthcoming 6G paradigm. However, one of the main characteristics of 6G resides in the ability of sensing [9], that is, being able to observe the conditions of the radio environment that could ameliorate communication systems to perform optimizations in terms of transmissions. LIS are a really useful tool for capturing the radio propagation environment as we have demonstrated this technology allows to represent the sense information as an image (holographic image), something that decreases complexity and permits to use image processing tools for studying the characteristics of the radio environment. What is more, the presented use case shows that machine learning algorithms, concretely computer vision ones, are a powerful tool to take into account when using an image-based approach. Besides, the high resolution image permits to work in really unfavourable SNR scenarios due to the macroscopic pattern obtained due to the large aperture. Future lines could be of interest, for example, user tracking, which could allow to perform optimizations such as beamforming. Future works will try to take advantage of the radio propagation conditions for detecting small angled deviations in the robot correct route as well as trajectories perpendicular to the LIS will be studied. The effect of mutual coupling due to small antenna spacings will be addressed to evaluate the impact on sensing performance [20]

. Besides, a further analysis of the images could be of interest for determining the optimal pattern of antenna elements deployment along the whole surface as well as antenna spacing to obtain the optimal LIS dimension. Also, deep learning algorithms for anomaly detection like autoencoders

[25] could be of interest for the aforementioned use case.


  • [1] F. Adib, Z. Kabelac, H. Mao, D. Katabi, and R. C. Miller (2014) Real-time breath monitoring using wireless signals. In Proc. 20th Annual Inter. Conf. Mobile Comput. Netw., pp. 261–262. Cited by: §I-C.
  • [2] D. Anguita, A. Ghio, S. Ridella, and D. Sterpi (2009) K-fold cross validation for error rate estimate in support vector machines.. In DMIN, pp. 291–297. Cited by: §III-A.
  • [3] S. Arikawa and Y. Karasawa (2014) A simplified MIMO channel characteristics evaluation scheme based on ray tracing and its application to indoor radio systems. IEEE Antennas Wireless Propag. Lett. 13, pp. 1737–1740. Cited by: §II.
  • [4] F. Chollet et al. (2015) Keras. Note: https://keras.io Cited by: §III.
  • [5] (Website) Note: https://www.altairhyperworks.com/feko Cited by: §II.
  • [6] S. F. Johnston (2006) Holographic visions: a history of new science. OUP Oxford. Cited by: §II-C.
  • [7] J. Joung (2016) Machine learning-based antenna selection in wireless communications. IEEE Commun. Lett. 20 (11), pp. 2241–2244. Cited by: §I.
  • [8] E. G. Larsson, O. Edfors, F. Tufvesson, and T. L. Marzetta (2014) Massive MIMO for next generation wireless systems. IEEE Commun. Mag. 52 (2), pp. 186–195. Cited by: §I.
  • [9] M. Latva-aho and K. Leppänen (2019) Key drivers and research challenges for 6G ubiquitous wireless intelligence (white paper). 6G Flagship research program, University of Oulu, Finland. Cited by: §V.
  • [10] D. Lee and G. B. Giannakis (2019) A variational Bayes approach to adaptive radio tomography. arXiv preprint arXiv:1909.03892. Cited by: §I-B.
  • [11] G. C. McGuire (2003) Using computer algebra to investigate the motion of an electric charge in magnetic and electric dipole fields. American Journal of Physics 71 (8), pp. 809–812. Cited by: §II-B.
  • [12] S. J. Pan and Q. Yang (2009) A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22 (10), pp. 1345–1359. Cited by: §III.
  • [13] Q. Pu, S. Gupta, S. Gollakota, and S. Patel (2013) Whole-home gesture recognition using wireless signals. In Proc. 19th Annual Inter. Conf. Mobile Comput. & Netw., pp. 27–38. Cited by: §I-A.
  • [14] T. S. Rappaport, R. W. Heath Jr, R. C. Daniels, and J. N. Murdock (2015) Millimeter wave wireless communications. Pearson Education. Cited by: §II.
  • [15] D. Sarkar, R. Bali, and T. Ghosh (2018)

    Hands-on transfer learning with python: implement advanced deep learning and neural network models using tensorflow and keras

    Packt Publishing Ltd. Cited by: §III.
  • [16] K. Simonyan and A. Zisserman (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. Cited by: §III.
  • [17] O. Stabler and R. Hoppe (2009) MIMO channel capacity computed with 3D ray tracing model. In 2009 3rd Europ. Conf. Antennas and Propag., pp. 2271–2275. Cited by: §II.
  • [18] R. Syms (1990) Practical volume holography clarendon. Oxford 19902, pp. 125. Cited by: §II-C.
  • [19] H. Wang, D. Zhang, Y. Wang, J. Ma, Y. Wang, and S. Li (2016) RT-fall: a real-time and contactless fall detection system with commodity WiFi devices. IEEE Trans. Mobile Comput. 16 (2), pp. 511–526. Cited by: §I-A.
  • [20] R. J. Williams, E. De Carvalho, and T. L. Marzetta (2019) A communication model for large intelligent surfaces. arXiv preprint arXiv:1912.06644. Cited by: §V.
  • [21] J. Wilson and N. Patwari (2010) Radio tomographic imaging with wireless networks. IEEE Trans. Mobile Comput. 9 (5), pp. 621–632. Cited by: §I-B.
  • [22] (Website) Note: https//www.altairhyperworks.com/winprop Cited by: §II.
  • [23] M. Zhao, T. Li, M. Abu Alsheikh, Y. Tian, H. Zhao, A. Torralba, and D. Katabi (2018)

    Through-wall human pose estimation using radio signals


    Proc. IEEE Conf. Comput. Vis. Pattern Recognit.

    pp. 7356–7365. Cited by: §I-C.
  • [24] Y. Zhao, N. Patwari, J. M. Phillips, and S. Venkatasubramanian (2013) Radio tomographic imaging and tracking of stationary and moving people via kernel distance. In 2013 ACM/IEEE Inter. Conf. Inf. Process. Sensor Networks (IPSN), pp. 229–240. Cited by: §I-B.
  • [25] C. Zhou and R. C. Paffenroth (2017) Anomaly detection with robust deep autoencoders. In Proc 23rd ACM SIGKDD Inter. Conf. Knowl. Discovery and Data Mining, pp. 665–674. Cited by: §V.