Skip to content

AWS incident response support: who actually answers when production breaks at 2 a.m.

AWS incident response support means three different things: AWS Support for their infrastructure, your own on-call for your workload, or an operations partner. Here is how to decide who owns the first fifteen minutes.

When production breaks at 2 a.m., “AWS incident response support” turns out to mean three different things, and confusing them is how outages stretch from minutes into hours. AWS Support answers for AWS infrastructure — a degraded host, an API outage, a stuck volume — and only at useful speed on the Business plan or above. Nobody at AWS will restart your application, roll back your deploy, or notice your database filled up. That part belongs to whoever holds your pager: your own on-call rotation, or an operations partner paid to be awake. The decision you are actually making is who owns the first fifteen minutes of your incidents.

What you are deciding

Three separate questions get bundled under “incident response,” and each has its own answer.

First: what does AWS itself cover? AWS Support plans, described at aws.amazon.com/premiumsupport, scale by response time and scope. Developer support is business hours and general guidance — useless for a production outage at night. Business support gives 24/7 access to engineers with a target of under one hour for a production system down. Enterprise On-Ramp and Enterprise tighten that to 30 and 15 minutes for business-critical cases and add a Technical Account Manager. Pricing is a percentage of your monthly AWS spend with plan-specific minimums, so the cost scales with the account. But the scope is AWS’s side of the shared responsibility line: their hardware, their managed services, their APIs. “My checkout is throwing 500s” is your incident unless you can show an AWS service caused it.

Second: who detects the incident? Support plans answer tickets; they do not open them. Without alarms wired to a paging path — the subject of the CloudWatch alarm baseline — the 2 a.m. failure is discovered at 8:45 a.m. by a customer.

Third: who responds to your workload? A real in-house rotation needs at least four people who can each diagnose the stack alone, or the rotation burns out the one person who can. Teams smaller than that are choosing between an operations partner and unstructured heroics, whether they admit it or not.

What to check first

  • Which AWS Support plan the account is on: Support Center > Support plans in the console. Many production accounts are still on Basic, which has no technical case support at all.
  • Whether a paging path exists — not a Slack channel, a path that makes a phone ring — and who is on the receiving end tonight.
  • Whether the person who would be paged can actually get in at 2 a.m.: working IAM credentials, MFA device at hand, VPN access, and the runbook location known.
  • Whether anyone is watching AWS Health for AWS-side events, so you can tell “we broke it” from “AWS broke it” in the first five minutes.

How to verify

Confirm what the support plan actually unlocks:

# Works only on Business, Enterprise On-Ramp, or Enterprise —
# an error here means the account has no 24/7 technical support
aws support describe-severity-levels --language en

# The AWS Health API has the same plan requirement
aws health describe-events \
  --filter eventStatusCodes=open,upcoming --region us-east-1

Then verify the human path with a drill, because it is the part that rots:

aws cloudwatch set-alarm-state --alarm-name prod-alb-5xx-rate \
  --state-value ALARM --state-reason "2am readiness drill"

Time three things: alert to acknowledgment, acknowledgment to logged-in-and-looking, and logged-in to first diagnostic conclusion. Those three numbers are your real incident response capability; everything else is paperwork.

The approach

  1. Define severities before the incident. Two levels are enough to start: SEV1, customers are impacted and someone gets woken; SEV2, degraded or at-risk and it waits for morning. Argue about the definitions in daylight, once.
  2. Match the AWS Support plan to the blast radius. Business support is the honest floor for any account where downtime costs real money — it is the cheapest insurance in the stack. Enterprise tiers make sense when 15-minute response and a TAM change outcomes, which is a spend and criticality question, not a pride question.
  3. Build the detection layer. Symptom-based alarms, two SNS topics (page versus ticket), a quarterly drill. Detection is upstream of every response decision and cheaper than all of them.
  4. Write the first-fifteen-minutes runbook. One page: how to check AWS Health, where the dashboards are, how to roll back the last deploy, when to open a Sev-1 case with AWS, who to escalate to. At 2 a.m. nobody thinks; they follow the page or they flail.
  5. Decide build versus buy for the rotation, honestly. Four or more capable engineers who accept the pager: build it, pay for it, and rotate fairly. Fewer than that: a monitoring and response partner is not an admission of failure, it is arithmetic.
  6. Close the loop after every incident. A blameless write-up with one action item that removes the failure class. Response quality compounds only if the same incident cannot happen twice.

When this is the wrong move

If the same incident recurs monthly, you have a reliability problem wearing an incident-response costume — fix the architecture instead of optimizing the ambulance. If the workload can honestly tolerate a night of downtime, paying for Enterprise support or a 24/7 rotation is spending money to protect nothing. And no support plan compensates for a single point of failure you already know about; AWS engineers responding in 15 minutes cannot un-lose the data on the one unreplicated volume.

Common mistakes

  • Believing AWS Support will debug application code. The shared responsibility model draws the line well above your stack traces.
  • Buying Business support and leaving detection unbuilt, so the 24/7 engineers are never called because nobody knows anything is down.
  • A “rotation” of one volunteer, which works until the week they are on a plane, and produces a resignation letter eventually.
  • Discovering during the outage that the on-call’s credentials expired, the MFA device is at the office, or the runbook lives in a wiki behind the VPN that is also down.

Keeping it under control

  • Run the 2 a.m. drill quarterly and track the three timings; regression there is an incident in its own right.
  • Review the support plan annually against current spend and criticality — accounts grow past their plan quietly.
  • Keep the first-fifteen-minutes runbook printed into the incident channel topic, not just in a wiki.
  • Rotate who runs the drill so the knowledge does not concentrate in one person.
  • Feed every real incident’s timeline back into the runbook within a week, while it is still true.

The plan, the pager, and the runbook only matter if a competent human is on the other end when the alert fires. If alerts fire into an empty room after hours, that gap is exactly what 24/7 Monitoring & Response closes — book a monitoring and response review.