Aborting Network Requests

Networking issues are very common. They happen more often than many people think. We are about to explore what happens when we simulate or create those same issues ourselves.

So, what can we do?

We can do many different things, but, in our case, we’ll start with something relatively simple. We’ll see what happens if we intentionally abort some of the network requests. We’re going to terminate requests and see how our application behaves when that happens. We’re not going to abort all the requests, but only some. Terminating 50% of requests should do.

What happens if 50% of the requests coming to our applications are terminated? Is our application resilient enough to survive without negatively affecting users? As you can probably guess, we can check that through an experiment.

Inspecting the definition of network.yaml

Let’s take a look at yet another Chaos Toolkit definition.

The output is as follows.

version: 1.0.0
title: What happens if we abort responses
description: If responses are aborted, the dependant application should retry and/or timeout requests
tags:
- k8s
- istio
- http
configuration:
  ingress_host:
      type: env
      key: INGRESS_HOST
steady-state-hypothesis:
  title: The app is healthy
  probes:
  - type: probe
    name: app-responds-to-requests
    tolerance: 200
    provider:
      type: http
      timeout: 5
      verify_tls: false
      url: http://${ingress_host}?addr=http://go-demo-8
      headers:
        Host: repeater.acme.com
  - type: probe
    tolerance: 200
    ref: app-responds-to-requests