Long Live The Image: Container-Native Data Persistence in Production

03/03/2021
by   Zheng Li, et al.
0

Containerization plays a crucial role in the de facto technology stack for implementing microservices architecture (each microservice has its own database in most cases). Nevertheless, there are still fierce debates on containerizing production databases, mainly due to the data persistence issues and concerns. Driven by a project of refactoring an Automated Machine Learning system, this research proposes the container-native data persistence as a conditional solution to running database containers in production. In essence, the proposed solution distinguishes the stateless data access (i.e. reading) from the stateful data processing (i.e. creating, updating, and deleting) in databases. A master database handles the stateful data processing and dumps database copies for building container images, while the database containers will keep stateless at runtime, based on the preloaded dump in the image. Although there are delays in the state/image update propagation, this solution is particularly suitable for the read-only, the eventual consistency, and the asynchronous processing scenarios. Moreover, with optimal tuning (e.g., disabling locking), the portability and performance gains of a read-only database container would outweigh the performance loss in accessing data across the underlying image layers.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/26/2017

A Comparative Analysis of XML Documents, XML Enabled Databases and Native XML Databases

With the increasing popularity of XML data and a great need for a databa...
research
11/26/2019

Cracking In-Memory Database Index A Case Study for Adaptive Radix Tree Index

Indexes provide a method to access data in databases quickly. It can imp...
research
11/01/2022

On Kubernetes-aided Federated Database Systems

Cloud computing has made federated database systems (FDBS) significantly...
research
10/28/2019

Machine learning on field data for hydraulic fracturing design optimization

This paper summarizes the efforts of the creation of a digital database ...
research
08/26/2023

Enhancement of database access performance by improving data consistency in a non-relational database system (NoSQL)

This study aims to enhance data consistency in NoSQL databases, traditio...
research
03/15/2023

A journey towards the most efficient state database for Hyperledger Fabric

The Hyperledger Fabric is well known and the most prominent enterprise-g...
research
06/28/2022

Manu: A Cloud Native Vector Database Management System

With the development of learning-based embedding models, embedding vecto...

Please sign up or login with your details

Forgot password? Click here to reset