Running One-Shot Experiments
Explore how to execute one-shot chaos engineering experiments inside a Kubernetes cluster by defining and managing Jobs. Learn to deploy experiments using container images, monitor pod status, inspect logs, and clean up resources efficiently.
For now, we’re going to focus on one-shot experiments that we’ll run manually, or through pipelines, or any other means that we might see fit.
Inspecting the once.yaml file
Let’s take a look at yet another Kubernetes YAML file.
The output is as follows.
---
apiVersion: batch/v1
kind: Job
metadata:
name: go-demo-8-chaos
spec:
activeDeadlineSeconds: 600
backoffLimit: 0
template:
metadata:
labels:
app: go-demo-8-chaos
annotations:
sidecar.istio.io/inject: "false"
spec:
serviceAccountName: chaostoolkit
restartPolicy: Never
containers:
- name: chaostoolkit
image: vfarcic/chaostoolkit:1.4.1-2
args:
- --verbose
- run
- /experiment/health-http.yaml
env:
- name: CHAOSTOOLKIT_IN_POD
value: "true"
volumeMounts:
- name: config
mountPath: /experiment
readOnly: true
resources:
limits:
cpu: 20m
memory: 64Mi
requests:
cpu: 20m
memory: 64Mi
volumes:
- name: config
configMap:
name: chaostoolkit-experiments
Kubernetes jobs
What matters is that we are defining a Kubernetes Job.
A Job creates one or more Pods and ensures that a specified number of them successfully terminate. As Pods successfully complete, the Job tracks the successful completions. When a specified number of successful completions is reached, the task (Job) is complete. Deleting a Job will clean up the Pods it created.
We can see from that description (taken from Kubernetes docs) that a Job is an excellent candidate to run ...