Back To Module Home

Distributed Systems: Building Software for the Real World

0% completed

Living in Production

Scope and Overall Development Lifecycle Pragmatic Architecture Quiz

The Exception That Grounded an Airline

Case Study: The Exception That Grounded an Airline The Change Window The Outage Consequences Postmortem Hunting For Clues The Smoking Gun An Ounce of Prevention Quiz

Stabilize Your System

Real World Problems Defining Stability Expanding System Lifespan Failure Modes Stopping Crack Propagation Chain Of Failure Quiz

Stability Antipatterns

Antipatterns Integration points Socket Based Protocols Network Connection The 5 AM Problem TCP With Firewall Socket Connection Failure HTTP Protocols Vendor API Libraries Countering Integration Point Problems Chain Reactions Avoid Failures Quiz

Failures And Blockages

Cascading Failures Users Heap Memory Sockets Expensive To Serve Non-standard Users Deadlock And Bad Users Malicious Users Blocked Threads Spot The Blocking Libraries Self-denial attacks Avoiding Self-Denial Scaling Effects Shared Resources Unbalanced Capacities Drive Out Through Testing Dogpile Quiz

Force Multiplier

Outage Amplification Controls and Safeguards Slow Responses Unbounded Result Sets Reasons for unbound result set Quiz

Stability Patterns

Timeouts Timeouts Side Effects And Latency problems Circuit Breaker Bulkheads Steady State Data Purging Log Files Fail Fast Let It Crash Handshaking Test Harnesses More About Test Harness and Failure Modes Decoupling Middleware Shed Load Create Back Pressure Governor Quiz

Launching An Online Store

My First Holiday Load Test

Taking the Pulse

Thanksgiving Day and Black Friday

Diagnostic Tests

Comparing options

Foundations

Physical Fundamentals NICs and Names Programming for Multiple Networks Virtual Machines in the Data Center Containers in the Data Center The 12-Factor App Virtual Machines in the Cloud Containers in the Cloud Quiz

Processes on Machines

Deployment and Runtime View Code Configuration Transparency Designing for Transparency Enabling Technologies and Logging Human Readability of Log Files and Voodoo Operations Instance Metrics and Health Checks Quiz

Interconnect

Interconnect Layer DNS Global Server Load Balancing with DNS Availability Of DNS And Load Balancing Software and Hardware Load Balancing Health Checks Demand Control Preventing Disaster Network Routing Discovering Services Migratory Virtual IP Addresses Quiz

Control Plane

Mechanical Advantage System Failure, Not Human Error Automation Goes Really Fast Platform and Ecosystem Development Is Production System-Wide Transparency Economic Value The Risk of Fragmentation and Logs/Stats What to Expose Configuration Services Provisioning and Deployment Services Command and Control The Platform Players Quiz

Security

Security Breaches Broken Authentication and Session Management Cross-Site Scripting Broken Access Control Security Misconfiguration Sensitive Data Exposure Cross-Site Request Forgery The Principle of Least Privilege Configured Passwords Waiting for Godot Quiz

Design for Deployment

Planned Downtime Automated Deployments Continuous Deployment Phases of Deployment Relational Database Schemata Schemaless Databases Web Assets Rollout Cleanup and Deploy Like the Pros Quiz

Handling Versions

Non-breaking API changes Gap Between Documentation and Actual Implementation Breaking API Changes Handle Others’ Versions Quiz

Case Study: Trampled by Your Own Customers

Countdown and Launch Load Testing Murder by the Masses The Testing Gap Aftermath Quiz

Adaptation

Convex Returns Process and Organization Platform Team Painless Releases Service Extinction Team-Scale Autonomy Beware Efficiency Quiz

System Architecture

Evolutionary Architecture Loose Clustering Explicit Context Create Options Splitting Incorporating Change Quiz

Information Architecture

Databases Messages, Events, and Commands Services Control Their Identifiers URL Dualism Embrace Plurality Quiz

Chaos Engineering

Production Environment Problems Antecedents of Chaos Engineering The Simian Army Adopting Your Own Monkey Injecting Chaos Targeting Chaos Disaster Simulations Quiz

Conclusion

...

/

Taking the Pulse

Taking the Pulse

Learn about the issues after launching an online site, load testing, GUI, Perl modules, and session counts.

Never forget the work

When we arrived on Wednesday night, I immediately set up my laptop in my parents’ home office. I can work anywhere I have broadband and a cell phone. Using their 3 MB cable broadband, I used PuTTY to log into our jumphost and start up my sampling scripts.

Looking up the GUI

During the run-up to the launch, I was part of load testing this new site. Most load tests deliver results after the test is done. Since ...