Skip to content

AWS Updated 2026-07-04

VPC route tables debugged: why traffic is not going where you think

When VPC traffic takes the wrong path, the cause is almost always route table association, longest-prefix matching, or a blackhole route. Here is how to debug AWS route tables methodically.

When traffic in a VPC goes somewhere unexpected — or nowhere — the explanation is almost always one of three things: the subnet is not associated with the route table you are staring at, a more specific route is winning over the one you added, or a route has gone blackhole because its target was deleted. AWS routing has no logs of its decisions, so you debug it by reconstructing the decision: find the table the subnet actually uses, then walk its routes by longest prefix match.

What’s happening

Every packet leaving an ENI is routed by exactly one route table: the one associated with the ENI’s subnet, or the VPC’s main route table if there is no explicit association. Within that table, AWS picks the most specific matching route — longest prefix wins, always. A /32 beats a /24 beats 0.0.0.0/0, regardless of the order routes were created or how they look in the console. When prefixes are equal, static routes you entered beat propagated routes learned from a virtual private gateway.

Two AWS-specific behaviours trip people up. First, the local route for the VPC CIDR exists in every table and keeps intra-VPC traffic inside the VPC; you cannot delete it (you can only override parts of it with more specific routes toward middlebox appliances, a deliberate and rare pattern). Second, routes do not disappear when their target does: delete a NAT gateway, a peering connection, or a transit gateway attachment and the route remains, flagged blackhole, silently dropping everything it matches.

Return traffic matters too. The destination subnet’s route table decides the reply path — asymmetric routing, where the reply takes a different route than the request, is a common outcome of half-updated tables and usually presents as connections that hang rather than fail cleanly.

What to check first

  • Which table is actually associated with the source subnet? Not the one named after it — the one the console shows under the subnet’s Route table tab.
  • Any blackhole routes? They are labelled in the console and in the CLI output; each one is a deleted target still being routed to.
  • Is a more specific route hijacking the traffic? Look for any prefix that contains your destination and is longer than the route you expect to match.
  • Does the return path exist? Check the destination subnet’s table for a route back to the source CIDR.
  • Are you in the right AZ’s table? Private route tables are typically per-AZ (one NAT gateway each); a fix applied to one AZ’s table does nothing for the others.

How to verify

Console: VPC > Route tables, filter by VPC, and read the Routes and Subnet associations tabs side by side. CLI:

# The table actually serving the subnet
aws ec2 describe-route-tables \
  --filters "Name=association.subnet-id,Values=subnet-0abc123" \
  --query 'RouteTables[].{Id:RouteTableId,Routes:Routes}'

# Every blackhole route in the VPC
aws ec2 describe-route-tables \
  --filters "Name=vpc-id,Values=vpc-0abc123" \
  --query 'RouteTables[].Routes[?State==`blackhole`]'

# Subnets with NO explicit association (inheriting main)
aws ec2 describe-route-tables \
  --filters "Name=vpc-id,Values=vpc-0abc123" \
  --query 'RouteTables[].Associations[?Main==`true`]'

To see whether packets are leaving at all, enable VPC Flow Logs on the source ENI and look for ACCEPT records with no reply traffic — the query patterns in VPC Flow Logs analysis apply directly. For a configuration-plane answer, run Reachability Analyzer between the source and destination (VPC > Reachability Analyzer); it names the exact route table and route that blocks the path. Route evaluation rules are documented in the Amazon VPC documentation.

The fix

  1. Fix the association, not the routes. If the subnet inherits the main route table or points at the wrong tier’s table, change the association (Route tables > Subnet associations > Edit). This is the most common root cause and the one people check last.
  2. Replace blackhole routes with live targets. Recreate the route pointing at the current NAT gateway, peering connection, or TGW attachment ID. The old ID in the route will never come back to life.
  3. Hunt the longer prefix. If a /24 toward a peering connection contains the host you are trying to reach through a VPN, the peering route wins. Either narrow it, remove it, or accept that the design says that traffic belongs on the other path.
  4. Repair the return path. Add the route back to the source CIDR in the destination subnet’s table. For peering and TGW paths both sides must be updated — one-sided routes produce timeouts, not errors.
  5. Apply the change to every AZ’s table. If you run one private route table per AZ, script the change across all of them rather than editing the one you happened to open.
  6. Re-test with a real connection, not just the console: nc -vz -w 5 10.1.2.3 443 from the source instance, and watch flow logs for the reply.

When this is the wrong move

If the route table is correct and traffic still dies, stop editing routes. Security groups and NACLs sit after the routing decision — a correct route with a blocked port looks identical to a missing route from the application’s point of view; the split is explained in security groups vs NACLs. And if you find yourself managing dozens of route tables to stitch many VPCs together with peering, the fix is not better route hygiene — it is moving the topology to a Transit Gateway hub-and-spoke.

Common mistakes

  • Debugging the route table the subnet is named after instead of the one it is associated with.
  • Forgetting that stopped-short prefixes still match: 0.0.0.0/0 to NAT will happily carry traffic you meant to send over a VPN if the VPN route is missing.
  • Editing routes in the main route table to fix one subnet, changing behaviour for every unassociated subnet in the VPC.
  • Assuming route propagation from a VPN is on — it is per-table and off by default.

Prevention

  • Explicitly associate every subnet; keep the main table as a local-route-only tripwire.
  • Alarm on blackhole routes with an AWS Config custom rule or a scheduled CLI audit — they only appear when something was deleted out from under production.
  • Manage route tables in Terraform or CloudFormation so drift shows up in a plan diff instead of an outage.
  • Name tables by tier and AZ (prod-private-rtb-ca-central-1a) so the association check takes seconds.
  • Run Reachability Analyzer as a post-change check after any peering, TGW, or VPN work.

Routing bugs feel mysterious because the data plane is silent, but the decision process is deterministic: one table per subnet, longest prefix wins, blackholes drop. Walk it in that order and the answer falls out. If nobody on your team truly owns the AWS environment end to end, that is the root of most of these problems — Stack Harbor runs AWS accounts daily under AWS Operations, and you can book an AWS review.