Cloudflare has reclaimed more than 100 terabytes of RAM across its global infrastructure, this time without adding hardware or reducing capacity. The savings came from a deceptively simple question: did its servers really need to maintain so many hash entries?
The answer was no.
Engineers working on Cloudflare’s Pingora Backend Router (PBR) found that its consistent-hashing system was maintaining as many as 100,000 hash points per server. By reducing that figure to 10,000 and making each entry more memory-efficient, Cloudflare eliminated substantial overhead while preserving the traffic distribution its network depends on.
At Cloudflare’s scale, a small software optimization became a 100 TB infrastructure gain.
Why PBR Needs Hash Rings
PBR helps determine which backend server should handle a cacheable request. Sending requests for the same content to the same server improves the chances that the requested object is already cached, reducing unnecessary retrievals and improving efficiency.
To accomplish this, Cloudflare uses consistent hashing. Requests and servers are mapped onto a virtual hash ring, allowing traffic to be distributed predictably while minimizing disruption when servers enter or leave the pool.
Each physical server appears at many virtual positions on that ring. More positions generally produce more even traffic distribution, which led Cloudflare’s implementation to use up to 100,000 points per server.
The problem was that those extra points were consuming a lot of memory for increasingly small benefits.
100,000 Points Were Hardly Better Than 10,000
Rather than assuming more hash points were always better, Cloudflare’s engineers analyzed how much distribution accuracy they were actually buying.
At 10,000 points per server, the distribution error was already around 1%. Increasing the ring to 100,000 points reduced it to roughly 0.3%.
That meant storing ten times as many entries produced only a modest improvement in balancing accuracy.
There was another complication. PBR uses 32-bit hash values, creating a finite number of possible positions. As more points were packed into that space, collisions became increasingly likely. At sufficiently large scales, adding more hashes could introduce additional error instead of continuing to improve distribution.
Cloudflare concluded that 10,000 points offered a much better balance between memory consumption and routing accuracy.
Dropping from 100,000 to 10,000 eliminated 90% of the entries.
Two Bytes Also Made a Difference
Cloudflare then looked at the size of each remaining entry.
The existing Rust structure used two 32-bit values: one for the hash and another identifying the associated server. Each point therefore occupied 8 bytes.
But the server index did not need 32 bits. A 16-bit value can represent 65,536 possibilities, far more than PBR needed within an individual data center.
Switching to a smaller index should have reduced each entry to six bytes, but normal Rust memory alignment could add padding and push it back to eight.
Cloudflare worked around that by representing each point with a compact six-byte structure. The result was a 25% reduction in memory per remaining entry.
Saving two bytes is meaningless when storing a handful of records. Across huge hash rings replicated throughout a global network, it becomes significant.
Combined with eliminating 90% of the points, the optimization reclaimed more than 100 TB of RAM.
Deploying It Was Harder Than Optimizing It
Changing the algorithm created another challenge: the cache itself.
A different hash ring can send a request to a different backend server. If Cloudflare had switched every server to the smaller rings at once, large numbers of cached objects could suddenly have been mapped to machines that did not already hold them.
That would have increased cache misses and potentially caused a surge of requests back to customers’ origin servers.
Cloudflare therefore allowed the old and new hash rings to coexist during the transition and rolled out the change gradually, data center by data center.
Engineers monitored memory consumption, backend selection, cache behavior, connection errors and origin traffic as the new configuration expanded.
It is a useful example of the difference between optimizing code and optimizing infrastructure. Changing a data structure can be relatively straightforward. Changing it safely while enormous volumes of internet traffic continue flowing through the system is considerably harder.
Cloudflare Has Now Done This Twice
The achievement is especially notable because it follows another 100 TB memory reduction from Cloudflare.
That earlier project focused on Big Pineapple, the caching platform behind services including 1.1.1.1, Gateway DNS and DNS Firewall. Cloudflare’s DNS infrastructure maintains more than 250 billion cache entries, making even tiny amounts of wasted memory expensive at scale.
Engineers reduced the typical memory footprint of a DNS cache entry from 953 bytes to 420 bytes, a 56% reduction that reclaimed roughly 100 TB of RAM while also improving cache performance.
The two projects attacked different problems.
The DNS work made each cached object substantially smaller. The latest PBR work determined that Cloudflare was maintaining far more hash points than necessary and then made the remaining entries smaller.
Together, the optimizations have reclaimed roughly 200 TB of RAM without installing additional memory.
Why 100 TB of Freed RAM Matters
For a network the size of Cloudflare’s, the benefit is not simply a smaller memory statistic.
RAM freed from one service becomes capacity available to another. It can support larger caches, additional workloads and future products while reducing pressure to expand physical infrastructure.
That has secondary benefits as well. Extracting more capacity from existing machines can delay server purchases and avoid the additional electricity, cooling, rack space and maintenance associated with expanding hardware.
The result illustrates an important characteristic of hyperscale infrastructure: tiny inefficiencies stop being tiny when they are repeated billions of times.
A few unnecessary bytes in a data structure or thousands of redundant hash entries may be invisible on an individual server. Across a global fleet, they can become tens or hundreds of terabytes of wasted memory.
Cloudflare did not find a revolutionary new type of RAM or install a faster generation of servers. Its engineers looked closely at an assumption buried deep inside the software, did the math and discovered that 100,000 entries were doing a job that 10,000 could handle almost as well.
The payoff was effectively 100 TB of new capacity from hardware Cloudflare already owned.
