On the Challenges and KPIs for Benchmarking Open-Source NFV MANO Systems: OSM vs ONAP

by   Girma M. Yilma, et al.

NFV management and orchestration (MANO) systems are being developed to meet the agile and flexible management requirements of virtualized network services in the 5G era and beyond. In this regard, ETSI ISG NFV has specified a standard NFV MANO system that is being used as a reference by MANO system vendors as well as open-source MANO projects. However, in the absence of MANO specific KPIs, it is difficult for users to make an informed decision on the choice of the MANO system better suited to meet their needs. Given the absence of any formal MANO specific KPIs on the basis of which a performance of a MANO system can be quantified, benchmarked and compared, users are left with simply comparing the claimed feature set. It is thus the motivation of this paper to highlight the challenges of testing and validating MANO systems in general, and propose MANO specific KPIs. Based on the proposed KPIs, we analyze and compare the performance of the two most popular open-source MANO projects, namely ONAP and OSM, using a complex open-source vCPE VNF and identify the features/performance gaps. In addition, we also provide a sketch of a test-jig that has been designed for benchmarking MANO systems.


page 1

page 2

page 5

page 8

page 9

page 10


A Versatile Dataset of Agile Open Source Software Projects

Agile software development is nowadays a widely adopted practise in both...

How do Developers Promote Open Source Projects?

Open source projects have an increasing importance on modern software de...

Towards Utility-based Prioritization of Requirements in Open Source Environments

Requirements Engineering in open source projects such as Eclipse faces t...

Kafka versus RabbitMQ

Publish/subscribe is a distributed interaction paradigm well adapted to ...

Fallout: Distributed Systems Testing as a Service

All modern distributed systems list performance and scalability as their...

Real-time Data Infrastructure at Uber

Uber's business is highly real-time in nature. PBs of data is continuous...

OpenSeqSLAM2.0: An Open Source Toolbox for Visual Place Recognition Under Changing Conditions

Visually recognising a traversed route - regardless of whether seen duri...