You may have heard of the infamous Chaos Monkey. What is about?

Chaos engineering is the discipline of experimenting on a system in order to build confidence in the system’s capability to withstand turbulent conditions in production.

To understand why we are here, we can introduce the reliance concept. It can be described as the ability of a given software to tolerate failures, while still ensuring adequate quality of service. In other word, we know issues are going to happen, we need to be prepared to face them.

Despite al the good planning, good development practices, tests and so on, it is not possible to prepare in advance for all possible issues, especially when these issues happen at the boundaries of different systems. Chaos engineering encompasses techniques aimed at meeting resilience requirements. It can be used to achieve resilience against infrastructure failures, network failures, and application failures; the latter includes resilience to unhandled exceptions.

Chaos Monkey

Chaos Monkey is a tool invented by Netflix in 2011, to test the resilience of its IT infrastructure. It works as a developer would never like to see. Intentionally, it disables components (e.g. computers) in Netflix’s production network to test how the remaining systems respond to the outage.

The name is explained in Antonio Garcia Martinez’s book “Chaos Monkey”:

Imagine a monkey entering a ‘data center’, these ‘farms’ of servers that host all the critical functions of our online activities. The monkey randomly rips cables, destroys devices and returns everything that passes by the hand [i.e. flings excrement]. The challenge for IT managers is to design the information system they are responsible for so that it can work despite these monkeys, which no one ever knows when they arrive and what they will destroy.

Chaos Monkey is now part of a larger suite of tools called the Simian Army designed to simulate and test responses to various system failures and edge cases.

After that, several other products have been developed, killing randomly kubernetes containers, or interrupting connectivity, and so on. The idea is essentially the same, test the system reliability by interrupting essential services, look what happens, collect valuable feedback, and prepare better for the worse!