Search⌘ K
AI Features

First Evidence and Blast-Radius Scoping on Linux

In troubleshooting on Linux, the initial minute is crucial for establishing the impact and scope of an issue, referred to as the blast radius. This involves articulating who is affected and the extent of the failure while gathering initial evidence through read-only checks, such as service states, logs, and resource pressures. A structured approach is necessary, starting with a clear scope statement and progressing through evidence collection without altering system states. The verification of fixes relies on successful request outcomes and improved error signals, emphasizing the importance of methodical troubleshooting.

Linux troubleshooting is a common part of DevOps interviews. You may be given access to a terminal, presented with a failing service, or asked to investigate an issue while explaining your decisions.

These scenarios test more than your familiarity with Linux commands. Interviewers want to understand how you assess impact, gather evidence, narrow down possible causes, and make changes without introducing additional risk.

This chapter explores how to approach Linux troubleshooting under interview pressure, from initial evidence collection and resource investigation to identifying failures and verifying recovery.

The first minute of a troubleshooting round shapes the interviewer’s impression. Before running commands, interviewers look for a clear statement of the impact, the affected scope, and the evidence that should be collected first.

What scope and first evidence mean

Scope is the current statement of who is affected and how far the failure reaches, also called the blast radius. A scope statement names the symptom, the affected service or users, and what is not yet confirmed. It describes impact and leaves the cause open.

First evidence is whatever can be collected without changing the system: read-only checks of service state, recent logs, host resource pressure, and the last change window. Reading first matters because a change can destroy the state needed to explain the failure. Restarting a service erases the failed process state and can hide a fault that recurs.

First evidence covers four areas:

  • Service state: Whether the unit (the systemd service definition) is running, failed, or restarting, shown by systemctl status for that unit.

  • Unit logs: Recent messages for the unit, shown by journalctl --unit with a time window that covers the alert.

  • Host pressure: Load, disk space, memory, and the busiest processes, shown by uptime, df -h, free -h, and ps or top.

  • Change clues: The last deployment or configuration change and when it ended.

Each signal has limits. A failed unit can be a symptom of an unavailable dependency, and a healthy unit can still return errors. uptime reports load averages over 1, 5, and 15 minutes that are not normalized for CPU count, so read them as a sign of pressure worth investigating, with the CPU count in mind. No single signal proves a root cause, so the next step is the cheapest read-only test that separates the leading hypotheses.

Scoping the order-service failures

Every lesson in this chapter uses one example: an order service that accepts order submissions and calls a payment gateway. The interviewer hands you this situation. An alert reports elevated errors from the service. A customer says order submissions are failing, and the last change window ended shortly before the alert. A stakeholder asks for an ETA. You do not yet know whether the fault is in the service, the host, or the payment gateway.

The scope statement for this situation is: “Order submissions are failing through the order API on this host; impact beyond this host and the payment gateway dependency is not yet confirmed.” That sentence directs the first action toward read-only evidence.

The timeline below places the alert, the customer impact, and the last change window in order; none of them yet shows a cause.

Start with read-only evidence to define scope
Start with read-only evidence to define scope

The first evidence for this service starts with the service state and its recent messages:

Shell-20
systemctl status SERVICE_NAME.service
journalctl --unit=SERVICE_NAME.service --since "15 minutes ago"

The systemctl status command may return output similar to this when the service repeatedly fails:

Output
SERVICE_NAME.service - Order service
Loaded: loaded (/etc/systemd/system/SERVICE_NAME.service; enabled)
Active: activating (auto-restart) (Result: exit-code)
Process: 4121 ExecStart=/opt/app/bin/server (code=exited, status=1/FAILURE)

Line 3 shows the unit restarting after an exit, and the status 1 on line 4 tells you the process itself failed. The journal then names the failure, for example lookup gateway: no such host.

A few minutes of collection records timestamps, the service state, a few relevant log lines, resource signals, and the change window. If the logs show connection failures near the alert, the next cheapest test may be a read-only reachability check toward the payment gateway; if storage is exhausted, inspect the affected filesystem and application write path before changing anything. The service is not restarted merely to see whether the alert clears.

Arrange the triage steps in the order you would take them, from the observed symptom to the next test, so evidence is collected before anything is changed.

Evidence-First Triage

Arrange the triage steps from symptom to the next test.

Scope the impact and blast radius

Collect service status and recent logs

Pick the next-cheapest hypothesis test

State the observed symptom

Check for recent change clues

Collect a quick resource snapshot

Do you find this helpful?

What the first minute establishes

The first minute gives the interviewer an indication of how you approach an incident. Can you explain what is failing and who is affected? Can you gather evidence before making changes? Can you communicate uncertainty without guessing at an ETA?

The same evidence-led approach applies when a fix is introduced. A restart or a healthy service status alone does not confirm recovery. The intended request must succeed, and the relevant error signal should improve over an agreed observation window.