Benchmarking Open-Source Static 3D Mesh Codecs for Immersive Media Interactive Live Streaming

Journal paper published by CERTH in IEEE JETCAS

Alexandros Doumanoglou, Petros Drakoulis, Nikolaos Zioulis, Dimitrios Zarpalas, and Petros Daras, Senior Member, IEEE

This work provides a systematic understanding of the requirements of live 3D mesh coding, targeting (tele-)immersive media streaming applications. We thoroughly benchmark in rate-distortion and runtime performance terms, four static 3D mesh coding solutions that are openly available. Apart from mesh geometry and connectivity, our analysis includes experiments for compressing vertex normals and attributes, something scarcely found in literature. Additionally, we provide a theoretical model of the tele-immersion pipeline that calculates its expected frame-rate, as well as lower and upper bounds for its end-to-end latency. In order to obtain these measures, the theoretical model takes into account the compression performance of the codecs and some indicative network conditions. Based on the results we obtained through our codec benchmarking, we used our theoretical model to calculate and provide concrete measures for these tele-immersion pipeline’s metrics and discuss on the optimal codec choice depending on the network setup. This offers deep insight into the available solutions and paves the way for future research.

Production Technology Seminar 2019 at EBU in Geneva

End of January, IRT participated in the 2019 edition of the Production Technology Seminar taking place at the European Broadcast Union in Geneva.

The event was dubbed “REVOLUTIONIZE MEDIA” and included different themes interesting for Broadcasters and Media Companies – specifically in the domain of production technologies and systems. Main topics included the transformation process of broadcast organisations, progress reports on new video and audio fundamentals, experiences from real-world UHDTV-HDR-HFR productions as well as pros and cons of cloud production technologies including their on-premise alternatives. Speakers presented how different technologies and trends are driving fundamental changes in how media is produced and consumed – including AI, new media experiences, immersive media tools and more.

IRT had the opportunity to perform a 5G-MEDIA survey during the event which helped us to gather meaningful feedback on our concepts from experienced industry stakeholders.

Come and meet our Team at the 5G-IA stand

5G-MEDIA will be demonstrating its results and solutions at the next Mobile World Congress, 25-28 February 2019 in Barcelona, with a dedicated space in the 5G-IA booth (7K17 in Hall 7).

Representatives from the project will be at the 5G-IA booth for the whole duration of the event to showcase 5G-MEDIA.

The demo will present the 5G-MEDIA platform in the context of the “Ultra High Definition media distribution over virtualized Content Delivery Network” use case. The focus of the use case is on enabling Media Service Providers to build flexible and adaptable media distribution service chains, made up of virtualized functions, and deliver UHD media contents while users are moving in a geographically distributed 5G network. In this perspective, the demo shows the orchestration and agile lifecycle management of media service chains including an UHD capable origin streaming server, media caches to be dynamically deployed in hierarchies at the edge of the 5G network (and serve media contents in proximity to the users), and a load balancer function for smart selection of delivering caches.

The demo will run on-site in a portable testbed, where a realistic scenario of combined core and edge clouds will be deployed to deliver the 5G-MEDIA media distribution service. The visitors will be able to use the 5G-MEDIA virtualized Content Delivery Network (vCDN) solution and consume UHD media contents in both HD TV screens and mobile devices, experiencing the flexibility and agility of the media distribution service.

The demo will showcase the integration and use of the 5G-MEDIA components for the optimization of the media service development and deployment:

  1. The 5G Apps and Services Catalogue for the creation and onboarding of the media service components, i.e. the Virtual Network Functions (VNFs) building the vCDN service;
  2. the 5G-MEDIA SVP for the instantiation and configuration of UHD media delivery services as NFV Network Services, making use of the Open Source MANO NFV platform;
  3. the 5G-MEDIA MAPE module for monitoring the running media services, detecting performance degradations (at both network and media service level) and react upon them to automatically optimize the service by scaling the media caches hierarchy and properly reconfiguring the load balancing function for maintaining UHD streams user perceived quality.

5G-MEDIA – 5G Service Platform Architecture for the Media Industry

On 18 January 2019, our colleague Madeleine Keltsch from IRT published an article “5G-MEDIA – 5G Service Platform Architecture for the Media Industry”.

Media applications are amongst the most demanding services in terms of resources, requiring huge network capacity for high bandwidth low latency streaming of audio-visual content. Recent technological advances in the 5G domain hold the promise to unlock the potential of the media industry by offering high quality media services through dynamic efficient resource allocation. Actual implementations are now required to validate whether advanced media applications can be realised benefiting from ultra-low latency, very-high bandwidth and flexible dynamic configuration offered by these new 5G networks. A truly integrated approach is needed that focuses on the media applications not only on the management of generic network functions and the orchestration of resources at the various radio, fronthaul/backhaul, edge and core network segments. The Horizon 2020 5G PPP Phase 2 project 5G-MEDIA funded by the European Commission leverages new options for more flexible, ad-hoc and cost-effective media workflows.

Read more…

5G-MEDIA @ the 9th Fokus Fuseco forum 2018

On 15 November 2018, our colleague from BitTubes presented the 5G-MEDIA project in the context of the Berlin5GWeek. The workshop was called “How Dynamic 5G Networks Unfold the Media Industry”.

The focus of research into 5G networks to date has been largely on the required advances in network architectures, technologies and infrastructures. Less effort has been put on the applications and services that will make use of and exploit the flexibility of 5G networks built upon Software Defined Networks/Network Function Virtualization (SDN/NFV) concepts in the network core.

The lecture presented a novel Service Virtualization Platform (SVP), called 5G-MEDIA SVP, architecture based on NFV and SDN to facilitate the development, deployment and operation of media services on 5G networks. The proposed SVP is being validated against three media use cases: an immersive Virtual Reality 3D gaming application, the remote production of broadcast content incorporating user generated contents, and dynamically adaptive CDNs for the intelligent distribution of Ultra-High Definition (UHD) content.

Igor Fritzsch – BitTubes GmbH, Germany

New 5G-MEDIA video online: The 5G-MEDIA Profiling tool (Demo)

The 5G-MEDIA Profiling tool provides load testing under various resource constraints on NSs which are deployed on the emulation platform. A variety of metrics can be monitored during these tests and these monitoring metrics can help service developers find bugs, detect bottlenecks or investigate problems in their media applications. The 5G-MEDIA profiling tool aims to automate big parts of this workflow and thus support service developers as much as possible. To achieve this aim, the 5G-MEDIA profiling tool creates one or more test containers to send different types of media traffics according to the use case.

The profiler gets the data from the time series database available in the 5G-MEDIA (Prometheus) and examines the association among the resources consumption and the overall traffic load served. In addition to the analysis results, simple visualisations are produced based on the Grafana tool.

Check the demo here
Do not hesitate to contact us for more information!

5G-MEDIA @ICT 2019

ICT 2018 took place in Vienna on 4-6 December 2018. This research and innovation event attracted 4800 visitors and focused on the European Union’s priorities in the digital transformation of society and industry. It presented an opportunity for the people involved in this transformation to share their experience and vision of Europe in the digital age.

In this context, our colleagues from Nextworks presented at the 5G-PPP booth the Use Case 3 demo “UHD media distribution over virtualized CDN”.

Do not hesitate to contact us should you need more information about our activities and next events!

5G-MEDIA team at NETAS recently presented at two different conferences

5G-MEDIA team members working on 5G-Media SDK recently published and presented two papers at two different conferences.

Dr. Refik Fatih Ustok of NETAS attended the IEEE Conference on Network Function Virtualization and Software Defined Networks in Verona, Italy between the 27th  and 29th of November.

This conference, shortly known as IEEE NFV SDN, is one of the flagship conferences of IEEE on network function virtualisation and network softwarisation. Refik Fatih presented their paper titled “Programming Tools for Rapid NFV-Based Media Application Development in 5G Networks.” In this paper, the authors explained the innovative aspects of 5G-MEDIA SDK as well as the programming tools such as FaaS Emulator and FaaS CLI Tools, validator, service monitoring and profiling which would simplify media application development through an all-in-one user interface (UI).

7th National Conference on Software Architecture

During the same week, Dr. Selcuk Keskin of NETAS also attended the 7th National Conference on Software Architecture which is a well-known conference in Turkey where all researchers working on software related topics share their research with the local audience.

Selcuk presented their paper in Turkish titled “5G Medya Endüstrisi İçin Uç İstemcilerden Buluta Sanallaştırma Yapısı” which means Edge-to-Cloud Software Virtualisation Structure for 5G Media Industry in English. In this paper, the authors briefly explained the 5G-MEDIA SDK and the all-in-one UI architecture in order to introduce the project to Turkish researchers.

5G-MEDIA talk @ Training School organized during the IEEE 5G and IoT Thessaloniki Summit 2018

Nick Zioulis – CERTH, Greece

On Monday, 22 of October 2018, our colleagues from CERTH were invited to talk about their ongoing developments in 5G-MEDIA at the Training School on Emerging Technologies for 5G and Internet of Things organized during the IEEE 5G and IoT Thessaloniki Summit 2018.

The talk was called “Exploring serverless service deployment in 5G for next generation media applications”. The agenda of the training school is available here.

5G-MEDIA Year 1 Achievements

As a 5G-PPP Phase 2 project, the 5G-MEDIA project aims at innovating media-related applications by investigating how these applications and the underlying 5G network should be coupled and interwork to the benefit of both. The 5G-MEDIA approach aims at delivering an integrated programmable service platform for the development, design and operations of media applications in 5G networks by providing mechanisms to flexibly adapt service operations to dynamic conditions and react upon events (e.g. to transparently accommodate auto-scaling of resources, VNF re-placement, etc.). Three use cases are going to be demonstrated: Immersive media and Virtual Reality, Smart Production and User-generated Content, and Ultra High Definition over Content Delivery Network.

5G-MEDIA Architecture

The main building blocks comprising the 5G-MEDIA architecture as shown in the figure include:

Application/Service Development  Kit (SDK)

Enables access to media applications development tools

Service Virtualization Platform (SVP)

Hosts the components related to the ETSI MANO framework, the 5G-MEDIA Catalogue designed to be NFV MANO platform-agnostic, the Media Service MAPE and the corresponding Virtualized Infrastructure Manager (VIM) and WAN Infrastructure Manager (WIM) plugins enabling the integration to different NFVI platforms

Network Function Virtualization Infrastructures (NFVIs)

Comprises the “Physical Layer” that provide computing resources by different operators and supporting different cloud technologies to host generic and media-specific VNFs depicted at the “Virtualized Resource Layer”

5G-MEDIA platform pioneers application of Function-as-a-Service (FaaS) to VNF management, complementing traditional VM based VNFs with FaaS based media specific functions, aiming at dramatically reducing development cycles and slashing operational costs to 5G-MEDIA users. The combination of the FaaS approach with the VNF packaging and the enablement of inserting FaaS VNFs in a typical VNF forwarding graph is one of the main innovation aspects of the proposed 5G-MEDIA approach. Another specific innovation of the 5G-MEDIA project is the NFV MANO platform-agnostic catalogue, which uses a novel generalized and extendible format for representing NSs and VNFs to enable the use of Virtual Functions in federated MANO systems (e.g. complement a domain’s catalogue of NSes and VNFs with items made available by other federated domains). In addition, the 5G-MEDIA platform integrates the Cognitive Network Optimizer (CNO) and the related monitoring components able to gather NFVIs, VNFs and applications metrics within the SVP. The Cognitive Network Optimizer relies on machine learning techniques and optimization policies management to adapt the deployment of VNF forwarding graphs seamlessly to continuously meet expected QoS requirements.

Tele-Immersive (TI) applications are immersive media network-based applications that enable the multi-party real-time interaction of users located in different parts of the globe, by placing them inside a shared virtual world. TI applications produce a large volume of heterogeneous data, thus, creating a challenging networking scenario. This use case requires high bandwidth (next-gen immersive 3D media), low latency streams between the players (establish the needed interaction) and smooth playback for the spectators. Within the scope of 5G-Media, this use case will demonstrate a network aware media application development through the deployment of media specific VNFs (e.g. transcoders), as well as showcase the potential of a FaaS application development model with specific VNFs being instantiated based on trigger/rule logic (e.g. replay generation) that increases modularity and overall offers more efficient architectural design. The Cognitive Network Optimiser will be involved in determining the best placement for all VNFs, considering the trade-off between performance and cost. The machine learning aspects of the smart engine will be used to forecast traffic demand according to a combination of anticipated and historical player and spectator locations.

Due to the steadily rising cost pressure, broadcasters are looking for new, low-cost and time-saving production methods, which include participatory and user-generated media archives in the production, also known under the term smart production. In 5G‑MEDIA it is planned to overcome these limitations by leveraging new options for more flexible, ad-hoc and cost-effective production workflows by replacing dedicated lines and hardware equipment with software functions (VNFs) facilitating (semi-) automated smart production in remote locations. Virtualised and flexible media services will reduce complexity for the user and ensure operational reliability (QoS, QoE). This use case is about the contribution of content in broadcast production quality from remote locations to a broadcasting studio. It therefore requires high bandwidth streams between the venue and the broadcaster for both fixed camera/studio locations and for mobile-generated content. Within 5G-MEDIA, the Cognitive Network Optimiser will be validated to enhance network performance (reserve bandwidth, prioritise traffic, define network paths etc.). In addition, Function as a Service (FaaS) capabilities will be used for the instantiation of the Cognitive Service VNFs for mobile content contribution sources.

Main purpose of this use case is to test access streaming of UHD media services through various personal devices, both fixed and mobile, while the user is on the move in the 5G network. The focus is on how UHD contents by a Media Service Provider (MSP) can serve users on the go and how the MSP can build media distribution service chains made of software defined media functions to properly serve users attached to the 5G network. This use case aims at orchestrating the lifecycle of service chains between an origin streaming server containing the UHD contents in various transcoded formats and the edge of the 5G network where media caches are to be dynamically instantiated to serve mobile devices. The target of this use case is to provide personalised view angles, with the possibility to compose UHD streams and move personal media while on the go. Key to the realisation of the use case is the possibility to implement high bandwidth streams between the streaming service (origin, replicas, transcoder and viewpoint servers) and users, in a flexible and dynamic way through the mechanisms of composition of virtualised network and media functions orchestrated by a NFV MANO layer. Within the context of 5G-MEDIA, the Cognitive Network Optimiser will adapt the media service chain related to the various media distribution flows based on end-to-end media quality control metrics and policies for scaling VNFs and optimising cache placement between the core and edge locations. The VNFs will be placed by the CNO according to predicted demand driven from a machine learning algorithm, analysing traffic demand. In addition, FaaS capabilities will be deployed for the programmatic selection of specific media streaming indicators and the activation/instantiation of e.g. new transcoding nodes at the edge to serve specific local demand from mobile users not previously transcoded.