Everything is Down!!

Discussion in 'The Veterans' Lounge' started by Zamiam, Aug 30, 2020.

  1. MasterMagnus The Oracle of AllHigh


    Of course not! (other than just jumping on my cell phone, duh)

    But I'm not a business with shareholders reading on CNN how their company failed today.
  2. Deddentd Journeyman

    I don't understand why anyone would trust Comedy News Networks for anything, since they don't really report the news.
  3. Smokezz The Bane Crew


    Read here for a for a technical post on why L3/CenturyLink caused the re-routing to NOT work. They were definitely the cause of the problem.

    https://blog.cloudflare.com/analysis-of-todays-centurylink-level-3-outage/
    svann and MasterMagnus like this.
  4. MasterMagnus The Oracle of AllHigh


    I had read that exact article earlier this morning. Thanks for posting, it's quite interesting.

    Yes I mentioned earlier Centurylink was (in my exact words) "the first domino to fall".

    The article mentions Cloudflare will have to wait for the Centurylink post mortem of the issue. And at several points reads a bit like a CYA from a company that has a bottom line and investors to consider.

    This happened in 2018 also, and 911 calls were even affected, both then and this time.

    Nobody is going to have 100% up-time. Hardware fails, things happen. The internet is designed to avoid single point failures to the best extent it can.

    All I'm saying is, when you're a big company you have back-up upon back-up.

    When Centurylink serves 911 calls, they need to have every precaution in place.

    When Cloudflare is bottlenecked by Centurylink, for whatever reason, they are incapable of providing their service. In the article they resort to "The old internet as a super highway" analogy to describe the single point failure. But the internet isn't supposed to work that way, exactly to avoid single point failures.

    When EQ has a single point failure in Cloudflare, I think they might be interested in knowing more about the details too.

    If Centurylink is solely to blame, I'm thinking a whole lot of people should be worried. Like what if this happened on (U.S.A.) election day?

    And if they aren't solely to blame, what went wrong at all the steps along the way?
  5. Jumbur Improved Familiar

    I wouldn't call it a "single point of failure", A lot of people successfully logged into everquest.com to complain(and some couldn't, like me), I still had (somewhat slow) access to "many" websites during that period.

    Obviously, their "rerouting strategy" helped mitigate the proplems somewhat. Otoh I aways thought it was problematic that a lot of DNS-servers was clustered around CERN(yes, they are cached around the globe, but still...) And a single submarine could probably "cut the internet performance in half" by destroying the right oversea cables...

    Im not blaming DBG for not having a secondary internet provider ready for these situations, after all it is "only a game". (911 Otoh should probably look into it though...) and who says Trump isn't aware that it could be an issue at elections...:p


    The internet nowadays, can still be disabled at war times(thank the governments for that, they all insists on having access to "kill-switches" :rolleyes: ), the ideal of a robust internet has been compromised ages ago.
    MasterMagnus likes this.
  6. Coagagin Guild house cat

    And when the attack are affecting the routers themselves prohibiting traffic reroutes because they are busy DDoSing the targets, you have a self failing system on two fronts. One, no one can stop the routers without manual intervention, leading to a reboot and flashing of the system. Never good. Or two, something new that Cloudflare has never seen leading to a number of 500 errors (something general and unspecific happened at the web server), a general collection box for the server to say "something happened that I (the server) do not understand. Rare but seen such.

    As for Cloudflare. Really freakin' good at what they do with little competition but a couple of other well known service providers. Best guess? Either a never seen before zero day or someone was stretching the patch cycle on routing to the point of someone being able to exploit the router OS, there are several but this too isn't unheard of.

    Meh. It will come out in the technical wash or press, which ever you prefer.The timing suggests the culprit was looking for self satisfying fame rather than financial unless you have the ransom note somewhere.

    - Coag
    MasterMagnus likes this.