For years, Cloudflare has provided a variety of services, including content delivery, DNS, and protection from DDoS attacks. Its services are widely used by many different companies and websites, though it’s also been criticized for serving as an enabler to online piracy, terrorist organizations (two of ISIS’ three forums in 2015 were guarded by Cloudflare), and other malcontents. Now, the company has announced that a serious flaw in its software may have served account logins and passwords inadvertently. Given how many websites use Cloudflare, that’s a big “Oops.” It’s being called “Cloudbleed” online, in reference to the massive “Heartbleed” bug discovered several years ago.
Cloudflare describes the problem as a buffer overrun, stating that its edge servers “were running past the end of a buffer and returning memory that contained private information such as HTTP cookies, authentication tokens, HTTP POST bodies, and other sensitive data. And some of that data had been cached by search engines.”
SSL private keys were not leaked (good), but the bug was active from February 3 to February 18. During that period, one out of every 3.3 million HTTP requests made through Cloudflare may have leaked data. As the company notes, one in 3.3 million is a very small number — but given the sheer volume of sites and the billions of HTTP requests flowing across the Internet on a daily basis, it’s not that small. Google was processing 3.5 billion search requests per day back in 2012 — so now imagine what traffic looks like now, and how often people might be hitting a Cloudflare-protected website without ever realizing they had done so.
What caused the issue? A bug that went undetected for years, but was itself blocked from leaking data by the way Cloudflare had configured its service. The company recently made some changes to its software, and those changes allowed the bug to begin leaking private data in a way it hadn’t previously done.
Here’s Cloudflare on the root cause of the problem:
The root cause of the bug was that reaching the end of a buffer was checked using the equality operator and a pointer was able to step past the end of the buffer. This is known as a buffer overrun. Had the check been done using >= instead of == jumping over the buffer end would have been caught. The equality check is generated automatically by Ragel and was not part of the code that we wrote. This indicated that we were not using Ragel correctly.
The Ragel code we wrote contained a bug that caused the pointer to jump over the end of the buffer and past the ability of an equality check to spot the buffer overrun.
Cloudflare notes that the nature of the bug means that accessing one site that used Cloudflare could leak information about a different site, and that passwords, API calls, URL parameters, cookies, and other sensitive information could have leaked. They are not aware of any sustained attempt to weaponize the bug, but that’s what you’d expect a company with a major security breach to say.
Gizmodo has published a list of known websites that use Cloudflare, including Patreon, Medium, Yelp, Uber, The Pirate Bay, Pastebin, Discord, Feedly, National Review, and 4chan. At least one popular game, League of Legends, also uses Cloudflare.
We recommend changing your passwords to avoid being put at risk, and to keep an eye on your accounts to watch for suspicious activity.