Skip to content

HAProxy rate limiting and tarpit

Stick-table-based rate limits in HAProxy — per-IP and per-key rules, the tarpit action to slow abusers, and the production policy we deploy in front of public APIs.

HAProxy implements rate limiting via stick tables — counters indexed by some request attribute (source IP, header, cookie) that you read in ACLs and act on with deny, tarpit, reject, or header injection. This article covers the production rate limit policy we put in front of public APIs: per-IP request rate, per-IP connection rate, a tarpit action for abuse, and the operational tradeoffs.

How to verify

For an existing rate-limit setup, the stick table itself tells you who is being throttled:

echo "show table fe_http" | sudo socat /run/haproxy/admin.sock - | head -20
echo "show table fe_http data.http_req_rate gt 100" | sudo socat /run/haproxy/admin.sock -
sudo journalctl -u haproxy -n 200 --no-pager | grep -i 'tarpit\|deny'
hey -n 1000 -c 20 https://api.example.com/test

show table fe_http data.http_req_rate gt 100 lists every source IP whose req rate has crossed 100 in the configured window. The hey line is a quick load generator; under it you should see your rate limit kick in.

What’s happening

A stick table holds counters per key. The data types you choose (http_req_rate(10s), conn_rate(5m), bytes_out_rate(1m), etc.) determine what HAProxy increments and over what sliding window. ACLs read those counters and decision rules act:

  • http-request deny — return an immediate 403 (or your configured status).
  • http-request tarpit — accept the request, hold it without responding until timeout tarpit elapses, then close. The attacker’s connection slot is consumed, your backend is untouched.
  • tcp-request connection reject — close the TCP connection before TLS, before HTTP parsing.
  • http-request set-header X-Throttled true — soft signal, the backend decides what to do.

Rate limit fairness is the production challenge. A naive http_req_rate(10s) gt 100 rule throttles a NAT’d CGI gateway as if it were a single user. The fix is to combine IP rate limits with header-based ones (per API key, per session cookie) and to whitelist known good corporate egress IPs.

The tarpit pattern is what makes HAProxy effective against brute-force tools: instead of an immediate 403 (which lets the attacker move on quickly), the tarpit holds the connection open without answering. Connection-bounded attackers (CLI brute forcers, slow exfiltrators) starve themselves on file descriptors.

The procedure

  1. Define the stick table at the frontend. Per-IP request rate over 10 seconds:

    frontend fe_http
        bind *:80
        stick-table type ip size 200k expire 30m store http_req_rate(10s),http_err_rate(10s),conn_rate(10s)
        http-request track-sc0 src
        default_backend be_app

    track-sc0 src ties incoming connections to the table by source IP, incrementing http_req_rate on each request.

  2. Add the deny rule. Anything over 100 requests in 10 seconds gets a 429:

    frontend fe_http
        bind *:80
        stick-table type ip size 200k expire 30m store http_req_rate(10s),http_err_rate(10s)
        http-request track-sc0 src
        acl too_many_req sc_http_req_rate(0) gt 100
        http-request deny deny_status 429 if too_many_req
        default_backend be_app
  3. Tarpit abusive sources. When the request rate is high AND a high error rate suggests guesswork:

    frontend fe_http
        bind *:80
        stick-table type ip size 200k expire 30m store http_req_rate(10s),http_err_rate(10s)
        http-request track-sc0 src
        acl abuse_req sc_http_req_rate(0) gt 200
        acl abuse_err sc_http_err_rate(0) gt 50
        http-request tarpit if abuse_req abuse_err
        default_backend be_app
    
    defaults
        timeout tarpit 5s

    The tarpit holds the connection for 5 seconds before closing. A brute-forcer scanning 10k passwords/sec slows to 12 passwords/minute per connection slot — usually fast enough to make them give up.

  4. Per-API-key throttling. Use the API key header as the stick key:

    frontend fe_api
        bind *:443 ssl crt /etc/haproxy/certs/api.example.com.pem
        stick-table type string len 64 size 100k expire 1h store http_req_rate(1m)
        http-request track-sc1 hdr(x-api-key)
        acl key_over_quota sc_http_req_rate(1) gt 600
        http-request deny deny_status 429 if key_over_quota
        default_backend be_api

    sc1 is a separate counter slot from sc0; you can track multiple keys per request (source IP on sc0, API key on sc1) and limit independently.

  5. Whitelist trusted sources. Known good IPs skip the throttle:

    frontend fe_http
        bind *:80
        stick-table type ip size 200k expire 30m store http_req_rate(10s)
        acl is_internal src 10.0.0.0/8 192.168.0.0/16
        acl is_partner src -f /etc/haproxy/partner-ips.lst
        http-request track-sc0 src unless is_internal or is_partner
        acl too_many sc_http_req_rate(0) gt 100
        http-request deny deny_status 429 if too_many
        default_backend be_app

    The track-sc0 src unless skips counter increment for trusted sources, so they cannot exhaust their share.

  6. Connection rate, not just request rate. Defense against connection floods:

    frontend fe_http
        bind *:80
        stick-table type ip size 200k expire 1m store conn_rate(10s)
        tcp-request connection track-sc0 src
        acl conn_flood sc_conn_rate(0) gt 50
        tcp-request connection reject if conn_flood
        default_backend be_app

    tcp-request connection reject closes the TCP socket before any TLS or HTTP work. This is the cheapest possible rejection.

  7. Inform the backend, do not block. Sometimes you want the backend to know a client is throttled without blocking:

    http-request set-header X-RateLimit-Bucket "%[src],%[sc_http_req_rate(0)]"

    The backend reads the header and decides — usually to slow its own response or queue.

Common pitfalls

  • A stick table without expire keeps growing — size 200k is the cap, but old entries take a slot until eviction. Always set expire.
  • The sc_http_req_rate(0) reads sticky counter 0; if you track on sc1, you read sc_http_req_rate(1). Mismatching the slot returns 0 silently and your rule never fires.
  • http_req_rate(10s) is a sliding window — the counter takes a few seconds to drop after a burst. A short bursty client that exceeds 100 in second 1 stays over the threshold for ~10 seconds.
  • Tarpit consumes HAProxy connection slots. Under a large attack, the tarpit pool can fill up and HAProxy starts refusing legitimate connections. Limit tarpit-eligibility to clearly abusive sources, not broad rate limits.
  • Rate limiting on source IP behind a CDN or upstream proxy throttles the CDN’s egress IP, not the real client. Use src from X-Forwarded-For (req.hdr(x-forwarded-for,1)), but only when the immediate hop is your own CDN.

Stack Harbor builds rate limit policies into the HAProxy edge as a baseline — per-IP req rate, per-API-key quota, abuse-flag tarpit, and a whitelisted partner list. The exact thresholds come from baseline traffic in the customer environment; we measure before we throttle. This is part of the abuse-protection layer we wire into our Managed Operations practice.