Automating chaos experiments in production

05/12/2019

∙

by Ali Basiri, et al.

∙

Netflix

∙

Distributed systems often face transient errors and localized component degradation and failure. Verifying that the overall system remains healthy in the face of such failures is challenging. At Netflix, we have built a platform for automatically generating and executing chaos experiments, which check how well the production system can handle component failures and slowdowns. This paper describes the platform and our experiences operating it.

READ FULL TEXT

Automating chaos experiments in production

Towards Runtime Verification via Event Stream Processing in Cloud Computing Infrastructures

Distributed joint dynamic maintenance and production scheduling in manufacturing systems: Framework based on model predictive control and Benders decomposition

How Bad Can a Bug Get? An Empirical Analysis of Software Failures in the OpenStack Cloud Computing Platform

Run-time Failure Detection via Non-intrusive Event Analysis in a Large-Scale Cloud Computing Platform

PREVENT: An Unsupervised Approach to Predict Software Failures in Production

When malloc() Never Returns NULL – Reliability as an Illusion

Intelligent Vision Based Wear Forecasting on Surfaces of Machine Tool Elements

Automating chaos experiments in production

Related Research

Towards Runtime Verification via Event Stream Processing in Cloud Computing Infrastructures

Distributed joint dynamic maintenance and production scheduling in manufacturing systems: Framework based on model predictive control and Benders decomposition

How Bad Can a Bug Get? An Empirical Analysis of Software Failures in the OpenStack Cloud Computing Platform

Run-time Failure Detection via Non-intrusive Event Analysis in a Large-Scale Cloud Computing Platform

PREVENT: An Unsupervised Approach to Predict Software Failures in Production

When malloc() Never Returns NULL – Reliability as an Illusion

Intelligent Vision Based Wear Forecasting on Surfaces of Machine Tool Elements