Notice that Test went down last week. https://forums.daybreakgames.com/eq/index.php?threads/test-server-downtime-tues-april-10.248555/ Augur noted the downtime and I would have expected the crash to not have happened because the developers took the time to TEST their deployment before launching it to the live servers. This is a common practice. Sometimes, you just can't simulate the live environment (our servers that we pay to play on) in a test environment. The crash on test may not have mimicked the full deployment impact that happened today. Whatever fix was put in place after Test crashed may not have solved the root problem. This happens. However, understanding the issue helps prevent similar issues from happening again. Don't be complaining that the servers are down, be complimenting the developers when the servers are up for extended amounts of time by writing comments like "You guys must have written some good notes about the deployment process this time! Keep up the good work!"
While it is a common practice, it does not mean it is the same. Development and pre-prod environments are never 100% the same as production. And that means no matter what tests your run there can always be issues. Any developer would love to have a duplicate of production, but its not cost effective to do that.
Test server back in COTF era was actually booming, with several very active guilds, including a current-content raid guild (I miss you Legions of Blackburrow). Then they decided to end the long-running practice of granting access to all expansions for free on Test Server, and it is now a shell of its former self. If they would not have made that change, it would likely be booming still, and would be a valid test environment.
You mean it's not just a magic box that developers speak to through a microphone and the RAM processor card just makes it work? Is this why bugs aren't addressed in minutes? It's all so enlightening.
A properly setup test environment should simulate everything but an actual live load, and even that can be simulated if you write tools to exercise the database based on a live sampling of typical DB queries. But let's be real here. Bugs like instances simply not working, and thousands of items being taken from peoples inventory because a change intended to apply to one server was applied to other servers, that's just bad programming/planning/testing. It's basics.
Exactly. But you left out deadline pressure. Most likely the date for the patch should have been moved when a bug of that severity was discovered to ensure that there was time for adequate testing. I'm more inclined to consider the short deadline the primary issue than just "bad programming" in some generic sense. All programmers engage in "bad programming" from time to time. It's not necessarily sloppiness. In this case, it could be that a developer found a problem that they legitimately believed was causative, and addressed it well. Yet if that wasn't the actual problem in the end, then the problem didn't get fixed and only adequate testing could uncover that the issue still existed. In particular, developer testing alone is never "adequate" because like all human beings developers have bias. "Knowing" the cause for something almost inevitably biases testing to conditions centered around that cause.
You're right about deadline pressure. Game companies quite often have utterly unrealistic schedules, then on top of that the game designers or producers come in and say "oh just this one little extra feature, it shouldn't take you long." At my last job one of these "producers" was a VERY attractive young woman. She'd be aloof from the devs until she needed something, then she'd literally sidle up to a lone dev and WHISPER to him what she needed. I know this due to the joys of open office floorplans so popular nowadays. She did this to a few deves, until one day her target, a younger nerdy dev too, spoke up loudly to her whispers, telling her she'd have to talk to his boss if she wanted any further changes made w/out a schedule change. That guy may have been young, but he was not stupid
I have a feeling a major reason its taking a while is mostly due to the fact its after hours. Unless they have a graveyard shift team working, they likely have few workers working on this fix and some probably pulling over-time.
I noticed complaints about instances being broken on test this last week. Gotta wonder if this has anything to do with that.
I think a lot of you are also forgetting about the staffing issue. I don't blame the devs, I blame Daybreak. Surely EQ is making enough to pay for a couple more people.
I would be happy just getting an update on whats happening with the server, how far along they've made it, any eta, ect ect
I use docker to duplicate production for allakhazam. It allows me develop and test everything out before pushing it out live. However that doesn't mean there can't be issues as you have pointed out. Sometimes the nut behind the keyboard can be the issue and sometimes those who implement things beyond your control can affect things as well.
I would say that EQ has millions of more lines of code than allakhazam. I remember my senior year at university working with only 5,000 lines of code in our project. We each had to write a compiler on a Dec Vax 11/730. Not fun. One company I worked for had a test server for thier business software. They would work on there, then push the code to an office with not much data flow. If it worked there, then they would push it to the other offices. Then things would pop up on my screen and I had to telephone the developers as I sat back in the computer room and monitored them via the console windows. I'm glad I don't have to do that anymore.
