Skip to content

Databases Updated 2026-05-29

Memcached caching system

How Memcached works as a distributed in-memory cache on Stack Harbor PaaS — client-server model, pooled memory across nodes, hash-table semantics, and enabling it in an environment.

Memcached is an open-source, distributed, high-performance in-memory object cache. It’s designed to speed up dynamic web applications by holding frequently-accessed data in RAM, so the application can read from Memcached instead of hitting a slower backing store (typically a database).

The mental model: a giant hash table, distributed across one or more Memcached nodes. Clients populate it and query it. When the table is full, older entries are evicted in least-recently-used (LRU) order.

Keys can be up to 250 bytes; values up to 1 megabyte. Both limits are configurable, but pushing them is rarely the right call — large values defeat the point of a cache.

Client-server model

Memcached uses a client-server model:

  • The servers maintain a key-value associative array in memory.
  • The clients populate the array (set, add, replace) and query it (get, gets).

The clients pick which server holds which key by hashing the key against the list of known servers. The servers themselves do not coordinate with each other — there is no cluster-level coordination, no replication, no failover. Each server is independent, and the clients shard keys across them.

This is the operational simplicity that makes Memcached fast and cheap. It’s also the constraint to keep in mind: lose a Memcached server, and the keys mapped to that server are gone. Memcached is a cache, not a primary store — your application has to be able to repopulate the cache from the database when it’s cold.

Pooled memory across nodes

The benefit of running multiple Memcached nodes is not redundancy — it’s a shared pool of memory the entire application tier can read from.

Consider two deployment patterns for a web tier with multiple application nodes:

  • Without Memcached. Each application node holds its own local in-process cache. Different application nodes hold different caches, with overlap and waste. A request that hits one application node doesn’t benefit from cached data on another. As you add application nodes, the total amount of caching grows, but the consistency across nodes gets worse.
  • With Memcached. All application nodes consult the same Memcached pool. A given cache key always lives at the same location, regardless of which application node serves the request. Adding application nodes does not duplicate the cache; adding Memcached nodes expands the pool. The two scale independently.

The second pattern is the right default for any horizontally-scaled web tier where cache hit rate matters.

Adding Memcached to an environment

From the topology wizard:

  1. Click Create Environment in the dashboard.
  2. In the Environment Topology dialog, pick your application server (for example, Tomcat).
  3. Enable Memcached in the same dialog.
  4. Set the Memcached node’s cloudlet limit (this caps the available cache memory).
  5. Name the environment (e.g. memcached-demo) and click Create.

After a couple of minutes the environment is up, with Memcached available at port 11211 on the Memcached node’s CNAME or private IP. Applications inside the environment can talk to it over the binary or text protocol — every major language has a Memcached client library (spymemcached and Xmemcached for Java, php-memcached for PHP, pymemcache for Python, memjs for Node.js).

Hit-rate intuition

A Memcached deployment is only valuable to the degree the application hits the cache more than it misses. Track the hit rate (cmd_get_hits / (cmd_get_hits + cmd_get_misses)) on each Memcached node — a hit rate over 80% is generally good, under 50% suggests the cache is too small for the working set or the TTLs are too aggressive.

Resize the Memcached node vertically when the working set outgrows the available memory.

See also