A multi-view video (MVV) is produced by simultaneously capturing a scene of interest from different angles with multiple cameras. Depth-Image-Based Rendering (DIBR) can be adopted to synthesize additional views which provide new view angles. Any view angle can be selected freely by an MVV user. MVV has been widely used in education, entertainment, medicine, etc. Compared to a traditional single-view video, an MVV usually has a large size, and hence yields a heavy burden for wireless networks. To improve transmission efficiency, usually, views are encoded separately and only the requested views are transmitted.
In [1, 2], MVV transmission in single-user wireless networks is studied. The antenna selection and power allocation are optimized to maximize the utility  or minimize the total transmission power. As view synthesis and multicast opportunities are not considered, the proposed solutions in [1, 2] cannot be directly extended to multiuser wireless networks. In [3, 4, 5, 6], MVV transmission in multiuser wireless networks is studied. To reduce energy consumption, [3, 4, 5, 6] propose transmission mechanisms which can exploit natural multicast opportunities. In particular, [3, 4, 5] focus on Orthogonal Frequency Division Multiple Access (OFDMA) systems. The power and subcarrier allocation are optimized to minimize the bandwidth consumption  or total transmission power [4, 5]. None of ,  and  achieves optimal power and subcarrier allocation, or utilizes view synthesis to further improve transmission efficiency. In , the authors consider Time Division Multiple Access (TDMA) systems and allow view synthesis at the server and users to create multicast opportunities. The total energy consumption minimization with respect to view selection and resource allocation is investigated.
In this letter, our goal is to investigate optimal MVV transmission in OFDMA systems. It yields a more challenging optimization problem than the one in  for TDMA systems, as the numbers of variables and constraints grow exponentially with the number of subcarriers and the joint discrete optimization of view selection and subcarrier allocation is highly nontrivial. First, we establish a communication model for transmission of an MVV to multiple users in an OFDMA system. Then, we formulate the minimization problem of the average weighted sum energy for view transmission and synthesis with respect to view selection and transmission power and subcarrier allocation. The problem is a challenging mixed discrete-continuous optimization problem with huge numbers of variables and constraints. Based on convex optimization and Difference of Convex (DC) programming, a low-complexity algorithm is proposed to obtain a suboptimal solution. Finally, advantages of the proposed suboptimal solution are numerically demonstrated.
Ii System Model
As shown in Fig. 1, a single-antenna server transmits an MVV to (1) single-antenna users in an OFDMA system.111The MVV model and view selection model are the same as those in . We present the details here for completeness. The set of user indices is denoted by . There are original views, denoted by . Between any two adjacent original views, there are evenly spaced additional views, where is a system parameter. The additional views can be synthesized via DIBR. Let denote the set of indices for all views. Suppose that the source encoding rates of all views are the same, which is denoted by (in bits/s).
The server stores all the original views in , and can synthesize any additional view in (if needed), based on its nearest left and right original views and , where and denote the greatest integer less than or equal to and the least integer greater than or equal to , respectively. Each user can synthesize any view based on two views in and , respectively, which are successfully received by him. Here, are system parameters, which reflect the qualities of synthesized views.
The view requested by user , denoted by , is known at the server. When one view is requested by multiple users, natural multicast opportunities can be exploited to improve transmission efficiency. When different views are requested by multiple users, multicast opportunities may be created based on view synthesis, referred to as view synthesis-enabled multicast opportunities , to improve transmission efficiency. An illustration example can be found in Fig. 1.
We study a certain time duration, over which multiple groups of pictures (GOPs) are transmitted and each user’s view angle does not change. The view transmission variable for view is denoted by:
Here, represents that view is transmitted by the server and otherwise. Denote . The view utilization variable for view at user is denoted by:
Here, represents that view is utilized by user and otherwise. Thus, to satisfy each user’s view request, we require:
Each user can use only the transmitted views, i.e.,
are referred to as view selection variables . Because of the coding structure of MVV, the values of cannot be changed during the considered time duration.
We consider an OFDMA system with subcarriers, denoted by . The bandwidth of each subcarrier is (in Hz). Assume block fading and consider an arbitrary slot. Let denote the power of the channel for subcarrier and user , where denotes the finite channel state space. The system channel state is denoted by . The server is aware of the system channel state .
The subcarrier assignment indicator variable for subcarrier and view under the system channel state is denoted by:
Here, represents that the server uses subcarrier to transmit view under , and otherwise. Suppose that each subcarrier is allowed to transmit only one view. Thus, we have:
The transmission power and rate for view on subcarrier under are denoted by and , respectively, where
Let . The total transmission energy consumption per time slot under is given by:
where and . Capacity achieving code is considered to obtain design insights. To make sure that all users have no stall during the video playback, the following transmission rate constraints should be satisfied:
where is the complex additive white Gaussian channel noise power for each subcarrier. Note that the communication model for transmission of an MVV to multiple users in an OFDMA system is more complicated than that in a TDMA system .
Let and denote the energy consumption for synthesizing one view per time slot at the server and user , respectively. Therefore, the weighted sum energy consumption per time slot under is given by:
Here, is the corresponding weight factor for users, and
denote the synthesis energy consumption per time slot at the server and all users, respectively. The average weighted sum energy consumption per time slot is given by:
where the expectation is taken over .
Iii Problem Formulation and Structural Analysis
In this section, the view selection, transmission power allocation and subcarrier allocation are optimized to minimize the average weighted sum energy consumption. In particular, consider the following optimization problem.
Problem 1 (Energy Minimization)
Let () denote an optimal solution of Problem 1.
Problem 1 is a two-timescale optimization problem. Specifically, view selection is in a larger timescale and adapts to the channel distribution; power allocation and subcarrier allocation are in a shorter timescale and are adaptive to instantaneous channel powers. Problem 1 has discrete variables, i.e., and continuous variables, i.e., , as well as constraints. There are two main challenges for solving Problem 1: discrete variables are involved and the numbers of variables and constraints are huge. It is worth noting that due to the couping between view selection and resource allocation, traditional optimization techniques for subcarrier allocation cannot be directly applied to solve Problem 1.
Define . Introduce , eliminate and , and separate and . Then, we obtain a related problem, which will facilitate the optimization.
Problem 2 (View Selection)
where is given by the following subproblem.
Problem 3 (Resource Allocation for and )
where , and .
We have the following result.
(sketch) First, by Lemma 2 in , we have . Thus, we can eliminate by replacing with , for all . Then, following the proof for Lemma 1 in , we can eliminate and simplify the constraints in (10), (11) and (12) to (16) for all . Finally, using instead of and separating and , we can obtain the equivalent formulation in Problem 2 and Problem 3.
Iv Suboptimal Solution
Iv-a Optimal Solution of Problem 3
In this subsection, an optimal solution of Problem 3 is obtained. By replacing the constraints in (13) with , we can obtain the relaxed version of Problem 3, which is convex and can be easily solved. By Lemma 1 in , we know that under a mild condition,222The condition can be easily satisfied when the channel conditions of users vary from each other . the optimal solution of the relaxed convex problem of Problem 3 is also an optimal solution of Problem 3. Note that in previous works on MVV transmission in OFDMA systems, optimal power and subcarrier allocation has not been obtained.
Iv-B Approximate Solution of Problem 2
Obtaining an optimal solution of Problem 2 using exhaustive search is not acceptable, when and are large. In this subsection, a low-complexity approximate solution of Problem 2 is proposed. Similarly, we use instead of for all and eliminate . In addition, we replace (11) and (12) with
Here, , and approximately characterize , and , and . The goal is to reduce the numbers of variables and constraints by imposing approximate constraints on the average resource allocation. Let , and . Therefore, we can obtain the following approximate problem.
Problem 4 (Approximation of Problem 1)
Let denote an optimal solution of Problem 4.
Problem 5 (Equivalent Problem of Problem 4)
where , represents the number of subcarriers allocated for the transmission of view , is the penalty parameter, the penalty function is given by:
The optimal solution of Problem 5 is denoted by .
Based on Lemma 2, we can solve Problem 5 (for a sufficiently large ) instead. As is concave and (3)-(5), (22)-(24) are convex, Problem 5 is a penalized DC problem. Thus, a stationary point of Problem 5 can be obtained by using the DC algorithm. Specifically, a sequence of convex approximations of Problem 5, each of which is obtained by linearizing the penalty function , are solved iteratively . Note that the convex approximation of Problem 5 has variables and constraints, which are much smaller than those of Problem 1. The DC algorithm can be conducted multiple times, with random initial feasible points of Problem 5, and the stationary point of Problem 5 that achieves the minimum average weighted sum energy consumption and zero penalty, denoted by , will be selected.
Iv-C Low-Complexity Algorithm for Problem 1
V Numerical Results
In this section, we numerically compare the proposed solution with two baseline schemes. In Baseline 1, view synthesis is adopted at the server but is not adopted at each user. In Baseline 2, view synthesis is adopted at each user but is not adopted at the server. Note that natural multicast opportunities can be utilized by both baseline schemes; Baseline 1 cannot create multicast opportunities based on view synthesis, but can ensure that no more than views are transmitted; Baseline 2 can create multicast opportunities based on view synthesis, but may transmit more than views. Both baseline schemes adopt optimal transmission power and subcarrier allocation using the method in . We adopt FFmpeg as the MVV encoder and MVV sequence Kendo as the video source. For all , assume . Suppose that views are randomly requested by the users in an i.i.d. manner, as in .333We omit the detailed view request model due to page limitation. Please refer to  for details. 100 realizations of and 500 realizations of are randomly generated. The average performance is evaluated.
Fig. 2 illustrates the weighted sum energy consumption versus the number of users and the Zipf exponent . Fig. 2 (a) shows that as increases, the weighted sum energy consumption of each scheme increases, due to the load increase. Fig. 2 (b) shows that as increases (i.e., view requests from the users are more concentrated), the weighted sum energy consumption of each scheme decreases, due to the increment of natural multicast opportunities. From Fig. 2, we can see that Baseline 1 outperforms Baseline 2, which reveals that naive creation of multicast opportunities usually causes extra transmission and yields a higher energy consumption. In addition, we can see that Algorithm 1 outperforms both baseline schemes, indicating the importance of the optimization of view synthesis-enabled multicast opportunities. The gains of Algorithm 1 over the two baseline schemes are significant at large or small , as more view synthesis-enabled multicast opportunities can be created.
Vi Conclusion and Future Works
In this letter, we studied the transmission of an MVV to multiple users in an OFDMA system. We exploited both natural multicast opportunities and view synthesis-enabled multicast opportunities to improve transmission efficiency. First, we established a communication model for transmission of an MVV to multiple users in an OFDMA system. Then, we optimized view selection, transmission power and subcarrier allocation to minimize the average weighted sum energy consumption. A low-complexity algorithm was proposed to obtain a suboptimal solution of the challenging problem. The proposed optimization method can be readily extended to tackle two-timescale optimal resource allocation problems in OFDMA systems. This paper opens up several directions for future research. For instance, the proposed multicast mechanism and optimization framework can be extended to design optimal multi-quality MVV transmission in OFDMA systems. In addition, a possible direction for future research is to design optimal MVV transmission in different wireless systems.
-  Z. Chen, X. Zhang, Y. Xu, J. Xiong, Y. Zhu, and X. Wang, “Muvi: Multiview video aware transmission over mimo wireless systems,” IEEE Trans. Multimedia, vol. 19, no. 12, pp. 2788–2803, Dec. 2017.
-  X. Zhang, Z. Chen, Y. Xu, Y. Zhu, and X. Wang, “Pom: Power-efficient multi-view video streaming over multi-antenna wireless systems,” IEEE Trans. Green Commun. Netw., vol. 3, no. 4, pp. 919–932, Dec. 2019.
-  J. Wu, Q. Zhao, N. Yang, and J. Duan, “Augmented reality multi-view video scheduling under vehicle-pedestrian situations,” in Proc. IEEE ICCVE, Oct. 2015, pp. 163–168.
-  Q. Zhao, Y. Mao, S. Leng, and G. Min, “Qos-aware energy-efficient multicast for multi-view video with fractional frequency reuse,” in Proc. IEEE CHINACOM, Aug. 2015, pp. 567–572.
-  Q. Zhao, Y. Mao, S. Leng, and Y. Jiang, “Qos-aware energy-efficient multicast for multi-view video in indoor small cell networks,” in Proc. IEEE GLOBECOM, Dec. 2014, pp. 4478–4483.
-  W. Xu, Y. Cui, and Z. Liu, “Optimal multi-view video transmission in multiuser wireless networks by exploiting natural and view synthesis-enabled multicast opportunities,” IEEE Trans. Commun., to be published.
-  C. Guo, Y. Cui, and Z. Liu, “Optimal multicast of tiled 360 VR video in OFDMA systems,” IEEE Commun. Lett., vol. 22, no. 12, pp. 2563–2566, Dec. 2018.
-  H. A. Le Thi, T. P. Dinh, and H. Van Ngai, “Exact penalty and error bounds in DC programming,” Journal of Global Optimization, vol. 52, no. 3, pp. 509–535, Mar. 2012.
-  X. Zhang, Y. Zhao, T. Tillo, and C. Lin, “A packetization strategy for interactive multiview video streaming over lossy networks,” Signal Processing, vol. 145, pp. 285–294, Apr. 2018.