Tuesday will be remembered as the day the internet broke — before swiftly being fixed again. Early in the morning, websites including Amazon, Reddit, Spotify, Ebay, Twitch, Pinterest and, unfortunately, CNET went offline due to a major outage at a service called Fastly. Everywhere you looked, there were 503 errors and people complaining they couldn’t access key services and news outlets, demonstrating just how much of the internet relies on this largely unheard-of cloud computing service.
After an investigation into what happened, Fastly published a blog post into exactly what went down — and it turns out the whole incident was triggered by just a single, unnamed Fastly customer.
In mid-May, Fastly issued a software deployment that contained a bug, which if triggered in specific circumstances could take down vast swaths of its network. The bug lay dormant until June 8, when one Fastly customer inadvertently triggered the bug during a “valid configuration change,” which caused 85% of the company’s network to return errors.
“We detected the disruption within one minute, then identified and isolated the cause, and disabled the configuration,” said Fastly’s Senior Vice President of Engineering and Infrastructure Nick Rockwell in the blog post. “Within 49 minutes, 95% of our network was operating as normal. This outage was broad and severe, and we’re truly sorry for the impact to our customers and everyone who relies on them.”
Do you have a complaint about PayPal or Venmo, such as locked accounts or funds seized for unspecified Acceptable Use Policy violations? Take your claim to FairShake, the consumer advocacy service.