Immersive Applications and Virtual Reality

Tele-Immersive (TI) applications enable real-time, multi-party interaction of users spread around the globe, by placing them inside a virtual world. With the ongoing Virtual Reality revolution, specialized hardware entering the consumer market and significant funds coming to VR related technologies, these next generation communication applications are now starting to emerge and are expected to take the networking world by storm. Quality of Service (QoS) and Quality of Experience (QoE) are top priorities in immersive media whereas availability and interaction between users are considered critical challenges that need to be met as they ensure a smooth user experience. High quality 3D reconstructions of users are created, usually in the form of time-varying meshes (TVM), which produce large volumes of heterogeneous data, thus, creating a challenging networking scenario. Although TVM data can be compressed via static mesh compression or techniques that exploit correlations of the data over time, compression schemes are not yet ready to support real-time applications. The volume of data produced by real-time TI applications is increasing dramatically, imposing limitations on the network transmission of those data, in real time, with current network technologies. As a result, the use of 5G networking technology appears now more than ever, as a necessity for real time TI applications not only in the gaming industry but also in other areas such as advertising or e-health. These have high requirements for (i) very high bandwidth, (ii) ultra-low latency, (iii) ultra-high reliability and (iv) broadband access in high crowd-density areas.

Scenario 1: Tele-immersive Participatory Media

In a TI game scenario multiple players from around the globe enter a 3D area constructed in real time. In order for the players to enjoy a smooth game experience inside the virtual world and be able to interact naturally with each other and the virtual world, ultra-low latency while transmitting heavyweight TI data is essential to deliver high QoE. Besides the geometry data, multiple textures, game state and the interaction data also need to be transmitted per frame and in real time. This introduces a set of heterogeneous data with different QoS/QoE expectations, where the game state and interaction data are of paramount importance to be transmitted with minimum latency.

Tele-immersive participatory media scenario

Utilizing appropriate real-time 3D reconstruction methodologies and compression, a fully configurable TI platform is needed. Parameters relating to the compression and quality of the representations needs to be modifiable, effectively achieving rich data volume parameterization. Setting the quality of the virtual “avatars” and the compression parameters accordingly will control the strain put on the network. A tight control of VNF Forwarding Graphs and micro-service prioritization is needed to dynamically select network nodes based on the aforementioned compression parameters and steer network traffic to the most suitable ones in order for optimal bandwidth efficiency to be achieved. This will allow the 5G network to be tested under different data payloads and consequently draw conclusions regarding the QoS and QoE.

Furthermore, fine-grained control of location-based or prioritization assets is needed due to the 3D assets of the virtual environment being dynamically delivered by the cloud instead of being local to each client. This can be accomplished by means of a VR engine deployed at strategically positioned and geographically distributed nodes. Remote users will then be serviced with terrains based on their country of residence, for example. Context-aware, content prioritization should be tailored to the needs and situation of individual users.

Scenario 2: Immersive Interaction Spectating

“The International” is one of the most important annual events in e-sports. It is the gathering of the best Dota 2 players, a competitive online game, battling for a prize pool of millions of dollars. This year, the tournament awarded the biggest prize pool in eSports history at over USD 20 million, breaking all previous records. Whereas augmented reality technology (AR) was implemented in the live feed showing the game’s characters on screen as if they existed in real life. Such scenarios are now becoming a reality and in this context, similar scenarios need to be supported by 5G networks and services. Viewers should be able to watch realistic representations of the players in real time not only on screens but on mobile devices too, interacting and playing with each other.

Immersive interaction spectating scenario

 

The flexibility of VNF needs to be utilized in the form of a rendering service in order to produce 2D views from 3D content to deliver lighter-weight data to the spectators. At the same time QoS/QoE parameters need to be monitored and in the case where quality levels are not satisfactory, the rendering service should be dynamically migrated to the edge of the cloud to improve network performance and satisfy the expected QoS.

Spectating can be accommodated on various platforms. Different platforms require different engagement prioritization and capabilities, with desktop computer spectators expecting full free viewpoint spectating. This can be realized by Network Slicing, which will allocate dedicated and prioritized VNFs and physical resources replicating the traffic but with an important distinction, the network slice will downscale the delivered content in terms of data volume and quality, in order to deliver it massively to the spectating users. As a result, after evaluating user and performance requirements, the right network resources could be assigned on the fly so that the required Quality of Experience (QoE) and other quality indicators can be met, a critical task for latency-critical applications, such as Tele-Immersion services. In this context, potential network traffic spikes would be dealt appropriately and timely by redirecting the traffic to the downscaling network slice.

5G will also enable a number of advanced functionalities to the aforementioned TI platform:

  • By taking the viewer’s viewpoint into consideration each time, the rendering service VNF will intelligently deliver only the required sub-set of the data (in terms of viewing direction) to the spectators’ mobile devices, effectively prioritizing bandwidth resources and reducing latency.
  • Another novel functionality that will be made possible is the introduction of instant replays for immersive media, where a cache VNF will be leveraged, buffering traffic for specific durations and enabling the on-demand request of a replay.

The streaming requirements in high-crowd density situation, such as in this case, makes the use of 5G technology the most appropriate, for the viewers to enjoy a seamless user experience.