Chaos engineering is the practice of deliberately introducing failures into a system — stopping a service, saturating a network interface, injecting latency into a dependency — to observe how the system responds and verify that its error-handling, degradation, and recovery behavior matches what was designed. For QA teams in 2026, chaos engineering is no longer exclusively a site reliability or platform engineering responsibility: the systematic design, execution, and documentation of failure scenarios fits naturally within QA's role, and the tooling has matured to make it accessible outside large engineering organizations.

What Chaos Engineering Is and Why QA Teams Are Taking Ownership in 2026

The core claim of chaos engineering is straightforward: if you do not test how a system responds to failure, you will learn about it in production when the failure is uncontrolled and the impact is real. The practice was formalized at Netflix under the name Chaos Monkey, which randomly terminated EC2 instances in production to force the engineering team to build applications that tolerated server loss. The principle has since broadened well beyond Netflix and well beyond production: chaos experiments are now run routinely in staging and pre-production environments as part of the normal release validation process.

In 2026, QA teams own chaos testing for the same reason they own other forms of non-functional testing: the verification of failure behavior is a testing activity that requires structured test design, documented hypotheses, observable assertions, and reproducible execution. These are QA competencies. The alternative — leaving chaos testing to platform teams who run informal experiments without documented assertions or pass/fail criteria — produces exploration rather than verification.

The specific failure modes that chaos engineering tests are those that cannot be adequately covered by conventional functional test suites. A functional suite verifies that the checkout flow completes when all services are available. A chaos test verifies that the checkout flow degrades gracefully when the payment service is unreachable — does it show a clear error, preserve the cart, not corrupt the order record, and recover when the service becomes available again? These are testable behaviors with verifiable outcomes. They belong in a test suite with documented assertions, not in an ad hoc experiment run once and never repeated.

For foundational context on how resilience testing fits within a complete QA program, see Astaqc's software testing services and the complete software testing guide.

Types of Failure Injection: Network, Resource, Dependency, and State

Chaos experiments target one of four categories of failure. Understanding the categories clarifies what each type of experiment can and cannot verify.

Network failure injection is the most broadly applicable category for application teams. Almost every service communicates with at least one dependency over a network, and almost every team has experienced a production incident caused by a dependency that became slow or unreachable. Testing whether the application handles 500ms of added latency, complete connection refusal to a third-party API, or a 503 response from an internal service is within reach of any team using a proxy-based tool like Toxiproxy.

Dependency failure injection is closely related and often uses the same tooling. The distinction is that dependency injection can also target the content of responses — an API returning the wrong status code, a database query returning empty results, or a message queue delivering messages out of order. These are faults that network tools do not cover; a service can respond promptly and still return incorrect data. Testing application behavior under incorrect dependency responses requires a mock or proxy layer that can modify response content, not just connection behavior.

State-based failure injection is the most application-specific and most valuable category for QA teams with access to staging environment data. Injecting an expired authentication token, removing a configuration key that the application expects to be present, or inserting malformed data into a message queue tests application behavior in conditions that are realistic but difficult to reproduce by conventional means. For teams that need support designing and executing structured chaos experiments, see Astaqc's test automation services and the AI in software testing guide.

How to Start Chaos Engineering Without a Dedicated Platform

The tooling gap is the most common reason QA teams do not run chaos experiments: enterprise chaos platforms require infrastructure investment, dedicated platform engineering support, and integration work that exceeds the capacity of a QA team operating within a normal product engineering organization. The practical starting point does not require any of these.

Toxiproxy is an open-source TCP proxy from Shopify that sits between your application and its dependencies in a test environment. You configure it to add latency, limit bandwidth, or refuse connections to specific upstream endpoints, then run your existing functional tests through the proxy. This requires no changes to the application code, no dedicated chaos platform, and no infrastructure beyond the ability to run Toxiproxy as a container alongside your existing test environment. The first chaos experiment for most teams is: run the core user journey tests through Toxiproxy with 1,000ms of added latency on the payment service, and verify the application behavior matches the expected degraded state.

WireMock is an HTTP mock server that can be configured to return specific response codes, delayed responses, or fault responses for any endpoint. Using WireMock as a dependency stub in a staging environment, you can simulate a third-party API returning 429 (rate limited), 503 (service unavailable), or a 200 with a malformed body — and run your tests against each scenario. WireMock integrates with most CI pipelines as a container and requires no infrastructure beyond a container runtime.

The Chaos Toolkit is an open-source framework for defining, executing, and recording chaos experiments as structured JSON or YAML documents. It has drivers for Kubernetes, AWS, GCP, and plain HTTP. Its value for QA teams is in the experiment-as-code format: chaos experiments defined in Chaos Toolkit are version-controlled, reproducible, and can be added to a CI pipeline as a step. This transforms chaos experiments from one-time manual activities into repeatable, auditable tests.

A realistic starting sequence for a team with no existing chaos engineering practice:

1. Select one critical external dependency — the most consequential third-party service or internal microservice your application calls.
2. Write down the expected application behavior when that dependency is unavailable: which UI state should appear, which data should be preserved, which operations should degrade gracefully.
3. Use Toxiproxy or WireMock to make that dependency unavailable in your staging environment.
4. Run your existing functional tests and observe which fail. Failures indicate either a real resilience gap or a test that needs to be updated to handle the degraded state.
5. Document the findings, fix the real resilience gaps, and add the chaos scenario to your regular test schedule.

This sequence produces a working chaos test in a day. It requires no new infrastructure purchases and no platform engineering involvement. For support with test environment setup and CI integration, see Astaqc's QA team service and the guide to outsourcing QA.

Integrating Chaos Tests Into a CI/CD Pipeline

Running chaos experiments manually in staging is a starting point, but the value compounds when chaos tests run automatically as part of the CI/CD pipeline. Automated chaos testing in CI means that resilience regressions — new code that removes a circuit breaker, introduces an unhandled timeout, or breaks a fallback path — are caught before deployment rather than in production.

The integration pattern depends on the tooling. For Toxiproxy-based experiments, the CI job starts a Toxiproxy container, configures the relevant failure conditions via Toxiproxy's API, runs the targeted test suite, and then tears down the proxy. The test suite itself uses standard functional test assertions — the chaos is in the environment configuration, not in the test structure. For Chaos Toolkit-based experiments, the CI job runs the chaos toolkit CLI with the experiment definition file; the toolkit executes the experiment, evaluates the hypothesis, and returns a pass/fail result based on the defined steady-state assertions.

Positioning chaos tests in the pipeline requires consideration of execution time and environment requirements. Chaos tests typically take longer than equivalent functional tests because they involve waiting for timeouts to occur, recovery periods to complete, or repeated attempts to observe expected degradation behavior. A chaos experiment that tests retry behavior with exponential backoff may need to wait 15–30 seconds for the retry sequence to complete. This makes chaos tests poor candidates for the PR-level gate (where fast feedback is the priority) but well-suited to a post-merge or nightly stage.

A practical pipeline structure for chaos testing:

• PR gate: Standard functional tests only. Fast, required for merge.
• Post-merge to main: Full functional regression suite plus a focused chaos suite covering the highest-risk failure scenarios (the 5–10 scenarios that, if they fail in production, cause the most severe user impact).
• Nightly scheduled run: Full chaos suite including slower experiments. Results feed into a stability dashboard. Failures alert the team but do not block in-flight deployments.
• Pre-release gate: Targeted chaos experiments for the features or services changed in the release. A release that modifies the payment service should include chaos tests for payment service dependency failure, regardless of whether those tests are in the standard post-merge suite.

For comprehensive CI/CD integration guidance, see Astaqc's test automation services and the software testing cost guide for cost modeling of chaos testing infrastructure.

Measuring Chaos Experiment Results: Hypotheses, Steady State, and Pass Criteria

Chaos experiments without explicit pass/fail criteria are observations, not tests. The distinction matters because an observation produces a report; a test produces a gate. For chaos engineering to function as a QA activity — one that can block a release, produce a defect ticket, or confirm a fix — each experiment needs a defined hypothesis and a steady-state definition that makes the result unambiguous.

The standard structure for a chaos experiment hypothesis is: "Given [the system in its normal state], when [this failure is injected], the system will [exhibit this specific behavior]." For example: "Given the payment service is running normally, when the payment service returns 503 for all requests, the checkout page will display a specific user-facing error message, the cart contents will be preserved in the session, and no order record will be created in the database." Each component of that hypothesis is testable with an observable assertion.

Defining steady state for a chaos experiment:

• Choose metrics that can be observed programmatically: HTTP response codes, UI element presence, database record state, log output, API response content.
• Establish the baseline values before injection: the application returns 200 on the checkout page, the cart has the expected items, no error messages are visible.
• Define the expected values during injection: the checkout page returns 200 but with an error message element present; no order record is created; the cart item count is unchanged.
• Define the expected values after recovery: the checkout page returns to its normal state within a defined time window after the dependency is restored.

Pass criteria for a chaos experiment are typically of two kinds. A hard criterion is a behavior that must be true for the experiment to pass — the application must not corrupt data, must not return an unhandled 500, must not silently fail with no user feedback. A soft criterion is a quality metric — the error message should appear within 5 seconds of the failure, the recovery should complete within 30 seconds. Hard criteria gate deployments; soft criteria feed the quality backlog.

Without documented criteria, a chaos experiment produces only a narrative: "we killed the payment service and the app showed a blank page." With documented criteria, the same experiment produces a defect: "expected error message element with id='payment-error' to be present within 3 seconds of failure; actual result was blank page with no error messaging, HTTP 500." The second form is actionable by a developer without any additional context. For guidance on defect documentation and QA team structure, see Astaqc's software testing services, Astaqc's QA team service, and the AI in software testing guide.

Frequently Asked Questions

Should chaos experiments be run in production or only in staging?

For most teams, chaos experiments should start in staging. Staging environments allow teams to run destructive experiments — crashing services, corrupting data, saturating resources — without risking real user impact or data loss. Once the team has built confidence in the experiment design, documented the expected system behavior, and confirmed the application handles the failure correctly, a subset of low-risk read-only chaos experiments can be extended to production to catch configuration drift that staging does not replicate. Running chaos in production before it has been validated in staging inverts the risk profile and provides little additional insight.

What is the difference between chaos engineering and fault injection testing?

Fault injection testing is a specific technique — deliberately introducing errors at defined points in code or infrastructure to verify error handling. Chaos engineering is a broader practice that includes fault injection but also encompasses emergent failure scenarios, hypothesis-driven experimentation, and steady-state monitoring. In practice, QA teams starting with chaos engineering typically begin with fault injection (using Toxiproxy or WireMock to inject specific faults) and evolve toward broader chaos experiments as experience and tooling mature.

How do chaos experiments relate to performance testing?

Chaos experiments and performance tests overlap when failure conditions include resource saturation — high CPU load, memory pressure, disk fill. A chaos experiment that saturates a service's CPU while running functional scenarios tests both the correctness of behavior under load and the resource handling of the service. Performance testing focuses on measuring throughput, latency, and error rates under load; chaos engineering focuses on verifying specific behaviors under injected failure conditions. The two practices are complementary and share infrastructure, but have different goals and pass criteria. See Astaqc's performance testing service for guidance on load and stress testing methodology.

How many chaos experiments should a team run regularly?

There is no universal target count. The right scope is determined by the number of critical failure scenarios the application needs to handle — the set of failures where incorrect handling would cause significant user impact or data integrity issues. A typical microservices application might start with 10–20 experiment scenarios covering the highest-risk service dependencies, critical resource contention cases, and the most common state failure modes. The suite grows as the team identifies new failure scenarios from production incidents, threat modeling, or architecture changes.

What skills does a QA team need to run chaos engineering?

The foundational skills are test design (structuring hypotheses and pass criteria), environment configuration (running containers and configuring proxies), and basic scripting (automating experiment setup and teardown). Teams with existing CI/CD experience and containerized test environments can start with Toxiproxy or WireMock with minimal additional learning. Advanced chaos experiments involving Kubernetes cluster-level failures, service mesh manipulation, or cloud provider fault injection require additional infrastructure expertise, which may be provided by a platform or DevOps team rather than QA directly.

How should a team handle chaos experiment failures that reveal architectural issues rather than code bugs?

A chaos experiment that reveals an architectural gap — missing circuit breaker, no retry logic at the API gateway layer, no fallback when a caching service is unavailable — produces a finding that is not a code-level defect but a design gap. These findings should be documented as technical debt items with a defined risk level and remediation priority. If the gap is severe — the application corrupts data or crashes entirely when a common dependency fails — the finding should be treated as a release blocker until the architectural fix is in place. See the guide to outsourcing QA for guidance on communicating non-functional findings to engineering leadership. Also visit Astaqc's testing documentation service for structured defect and finding documentation.