T-Mobile outage caused by configuration error, not a DDoS attack

Rumors began circulating on Twitter yesterday that the U.S. was under a massive DDoS attack. It turned out to be an outage on T-Mobile caused by a configuration error.

It started with a tweet by an ‘Anonymous’ Twitter account that showed a “DDoS attack” map indicating that the USA was under a massive  DDoS attack allegedly from China.

It started to spiral downward with people quickly retweeting the information to the point that a US congressman retweeted the rumors of the attack.

DDoS (Distributed Denial of Service) attacks occur when servers and cyber-systems are overloaded with more traffic and network requests than they are equipped to handle.

To evade detection and security measures, such as automatic IP blocks, DDoS attackers typically use a large army of bots, different IP addresses, and networking tactics to make it harder for systems to tell malicious traffic from legitimate individual requests.

These attacks result in the target systems being unable to respond or crashing altogether, therefore causing outages.

However, industry experts and security researchers immediately became skeptical whether these outages were indeed resulting from a DDoS attack, or was it a case of simple network change gone wrong?MalwareTech✔@MalwareTechBlog

So far no provide other than T-Mobile has declared they’re having issues (and T-Mobile hasn’t said anything about DDoS), but in 1 hour we’ve already gone from an Anonymous account tweeting a meme pewpew map to senators chiming in? Can we maybe pump the brakes and wait for info?2,3096:35 AM – Jun 16, 2020Twitter Ads info and privacy576 people are talking about this

Cloudflare CEO, Matthew Prince also tried to quell the rumors of DDoS attack via a series of tweets:

“There’s a lot of buzz right now about a “massive DDoS attack” targeting the US, complete with scary-looking graphs (see Tweet below). While it makes for a good headline in these already dramatic times, it’s not accurate. The reality is far more boring.” 

“It [started] with T-Mobile. They were making some changes to their network configurations today. Unfortunately, it went badly. The result has been for around the last 6 hours a series of cascading failures for their users, impacting both their voice and data networks,” Prince tweeted.

In the same Twitter thread, Prince posts detailed infographic generated by Cloudflare that shines a light on the “boring” reality of internet traffic.

An “outages” mailing list, used to report “failures of major communications infrastructure components having significant traffic-carrying capacity,” also only saw problems stemming from the T-Mobile network.

Needless to say, in these times of heightened anxiety from the pandemic and protests, the outage left over 110,000 T-Mobile users unable to make or receive calls, starting around 12 PM EST on June 15th.

T-Mobile’s President of Technology, Neville Ray tweeted, “Our engineers are working to resolve a voice and data issue that has been affecting customers around the country. We’re sorry for the inconvenience and hope to have this fixed shortly.”

“Teams continue to work as quickly as possible to fix the voice & messaging problems some are seeing. Data services are now available & some calls are completing.  Alternate services like WhatsApp, Signal, iMessage, Facetime etc. are available. Thanks for your patience.”

As of today, just a few hours ago, Ray has confirmed the issue is now resolved: “Voice and text services are now restored. Thank you for your patience as we fixed the issues. We sincerely apologize for any and all inconveniences.” 

This outage and the speed of its resolution certainly did not sit well with some of the telecom’s customers: 

Outages like these reinforce our dependence on technology and the digital realm, and the inevitable responsibility companies have in ensuring their uptime, in addition to securing them from the latest threats and attacks.