Major Internet Outage Fully Resolved After Cloudflare Technical Failure Affects Websites Worldwide

Major internet outage yesterday caused by Cloudflare technical glitch affected websites worldwide for over 3 hours. Issue now fully resolved.

Major Internet Outage Fully Resolved After Cloudflare Technical Failure Affects Websites Worldwide
Image: Cloudflare

A technical glitch at internet infrastructure giant Cloudflare caused widespread disruption to websites and online services yesterday, leaving users across Ireland and globally facing error pages for over three hours.

The outage began at approximately 11:20 GMT yesterday morning and affected millions of websites that rely on Cloudflare's services, including major platforms and businesses. The issue was largely resolved by 14:30 GMT, with all services fully restored by 17:06 GMT.

Cloudflare, a company that helps protect and speed up websites for millions of customers worldwide, has confirmed the disruption was caused by an internal software bug and not by any cyber attack or malicious activity.

What Happened

The problem stemmed from a configuration file used by Cloudflare's bot management system, which helps protect websites from automated attacks and spam. During routine database permission updates, a query began producing duplicate entries in this file, causing it to more than double in size unexpectedly.

The software managing traffic across Cloudflare's network had a built-in limit on how large this file could be. When the expanded file was distributed to servers worldwide, it exceeded this limit and caused the system to crash.

Users trying to access affected websites were greeted with HTTP 500 error messages, a technical way of saying "something went wrong on the server". The issue affected core web traffic, login systems, and various other online services that depend on Cloudflare's infrastructure.

How It Was Fixed

Cloudflare's engineering team initially suspected the symptoms might indicate a large-scale distributed denial of service attack, which delayed diagnosis. Once they identified the root cause, they stopped the distribution of the problematic configuration file and replaced it with an earlier, working version.

Matthew Prince, Chief Executive Officer of Cloudflare, issued a public apology:

"An outage like today is unacceptable. On behalf of the entire team at Cloudflare, I would like to apologize for the pain we caused the Internet today."

The company has acknowledged this was their worst outage since 2019 and has committed to implementing multiple safeguards to prevent similar failures, including better handling of configuration files, additional emergency controls, and improved error management systems.

Cloudflare's status page also experienced unrelated technical difficulties during the incident, which initially added to concerns about a potential coordinated attack before engineers confirmed the issues were separate.

Impact on Users

The outage affected a wide range of services, from websites and mobile applications to business systems and security tools. Any website or service using Cloudflare's network experienced disruption, though the severity varied depending on which Cloudflare systems they relied upon.

For most internet users, the outage meant being unable to access favourite websites or online services for several hours. Businesses relying on Cloudflare experienced lost revenue and customer frustration during the downtime.

The company has published a detailed technical breakdown of the incident and the steps being taken to prevent future occurrences.