A senior site-reliability engineer is performing a failure-mode analysis.
INPUTS:
- Code: [Paste code]
- External dependencies: [List—third-party services, databases, queues, APIs, etc.]
TASK: Assess the code's behavior in the event that internal, network, storage, dependency, or input failures occur.
Examine the checklist:
- Error propagation and exception handling
- Retry logic, which includes exponential backoff and retry storms
- Circuit breakers and timeouts
- Fallback actions
- Idempotency
- Cleaning up resources
- Transaction safety and partial failures
- Error messages that are visible to users
- Observability and logging
- The conduct of recovery
PARTICULARLY:
- Errors that are swallowed or silently disregarded
- Erroneous or dangerous retry patterns
- Errors that end consumers are overexposed to
- Failure routes that result in an inconsistent state for the system
OUTPUT: Describe the risk, offer a suggested solution, and explain the expected/correct behavior during each failure scenario that has been discovered.