Update on the Briggs Server

Discussion in 'Official News and Announcements' started by RadarX, Apr 11, 2014.

Thread Status:
Not open for further replies.
  1. RadarX Moderator

    We realize the login issues our players in Sydney experience has been aggravating (it’s been to us as well) and although it may not be immediately evident, we’ve been trying various approaches to deal with it recently. While the situation has improved, unfortunately none of these have conclusively solved the problem. It boils down to the connection from our data center in Sydney having performance issues getting to our Las Vegas data center both in terms of packet loss and aggregate latency. It is expected that transoceanic connections have packet loss; so we have various network layers to deal with that and compensate.

    Recently the situation has gotten worse and reached a tipping point that has overwhelmed the networks ability to compensate for loss (and latency plays a factor as well, and has also been variable). We have tried various configurations to force routing to one path or another, and some of them have shown some progress but then later end up with similar performance issues.

    This doesn’t affect moment to moment game play once you are logged in, as that is entirely local to AUS/Sydney of course. But your character and account have a lot of data that is sent back and forth from wherever you are playing, so it does come into play for login, which is where you are seeing the problems. That is centralized so players everywhere in the world can compare themselves on Leaderboards, chat back and forth, and various other internal systems which require data from all characters to work properly.

    We built a new cache layer process specifically for Sydney to help eliminate some of the back and forth communication and just keep it local when there was no need to send it. This actually did a great job on the data it felt safe to cache, but ended up having too many dependencies on data it did NOT think safe to cache, and that still suffered from the same problems resulting in long load times.

    We’re currently looking at a completely different link configuration to try to address the fundamental problem. Past that there are other approaches we will attempt in terms of caching data and recovering more quickly from loss conditions. All of these take time and in the case of external route configurations are not always in our direct control. Thanks again for all of your patience during this work, and please know we’re going to be doing everything required until its resolved.
    • Up x 22
  2. dBus

    Thank you very much for this. I appreciate you guys going out of your way to work on this, and to let us know exactly why it's been having issues.

    Networking problems can be a massive pain. Maybe you guys need some sort of Satellite based WAN caching technology if you are trying to eliminate problems caused by latency? (Satellite based, as in technology designed to allow satellite links to work with high latency, like Riverbed, etc. Not actual satellites.)

    But, I'm sure your network guys will be working on it.

    Thanks again.
    • Up x 3
  3. Talboturbo

    Well, at least people know know you are working on it. Cheers guys! I look forward to you guys fixing it for the Sydney dudes ripping their hair out, or something.
    • Up x 1
  4. NovaAustralis

    Good to know you are working on it!
    Thanks for the detailed update.
  5. RyanGUK

    What was the reason for the situation getting worse? Was it after a patch, some maintenance or something external to the company?
    • Up x 2
  6. Cheesburgers

    Thanks, guys! It's great knowing that you're trying to solve these issues! Keep us posted :)
    • Up x 2
  7. Sen7rygun

    Thanks very much for the update guys. It's good to get some perspective on the technical issues that are creating this issue.

    Out of interest, how much data does an average character need to transmit to your login servers in Las Vegas? Is it 500kb, a couple of megabytes? I knew there is some inherent packet loss when sending data across oceanic transmission cables but I had no idea it was that big an issue.
    • Up x 1
  8. chrisarn94

    I'm curious as to why you can't keep the data you need to log into Briggs local. Why does there need to be a connection to Las Vegas?
    • Up x 1
  9. Kn1fey

    I would assume because Station Accounts are global to multiple server and games
  10. KJ Mac 50

    This is going to make me look stupid but I keep hearing that there is a test server how can I go on it if there is one?
  11. BluescalesNZ

    Glad to hear that the problem is being worked on. Hope you guys can get it sorted out soon. Don't forgot though that we Kiwi players use that server too, so it's not just Australians having to deal with the problem :p
    • Up x 2
  12. Sen7rygun

    I think you guys are coming through the Sydney exchange before the data goes offshore so you're getting caught in the same trap we are.
    • Up x 1
  13. Admiralty

    Look down the forum a bit for a section dedicated to the Test Server.
  14. BluescalesNZ

    Yeah, that'll be it. Do you get massive lag spikes every 10-20 minutes? I've had my ping shoot up as high as 71K on occasion.
  15. Sen7rygun

    I get occasional hitches and desynchronisation at around the same intervals but my hitching is generally limited to a couple of seconds and any desync maybe 7 to 10 seconds before it all rubber bands back into shape and starts behaving itself again.
  16. Mikromancer

    thanks for the hard work guys! it's a b**** having to deal with the massive login times, but it could be much worse, so I'm just glad I can play in the first place :)
    • Up x 1
  17. Talthos

    You know, why can't they just have login centers for each individual region locally? Like one on the west coast somewhere for US West, east coast for US East, maybe England for EU, etc. Is this really that hard to do?
  18. chrisarn94

  19. SaltNPepper

    Thanks for the update on the situation with login.
  20. LagLight

    This is how all developer problem explanation posts should be done, even if it does not offer a direct solution to a problem.

    I hate being kept in the dark by most developers about the specifics of important game issues, or even the developer's stance or point of view on popular game topics and discussions.
    • Up x 4
Thread Status:
Not open for further replies.