Testing Standards.....

Discussion in 'Testing Feedback' started by CrappyHeals, Jun 30, 2017.

  1. CrappyHeals Devoted Player

    Now that things are getting closer to final balance we really need some testing standards. I see way to many people saying this and that, posting off numbers or testing in ways that won't give you correct data and if we try to balance things around incorrect data this is gonna be a s*** show.

    Everyone is testing with different numbers of skill points, parser times and all kind of other variables. We need to get it narrowed down if we want to get this right.

    We need a set number of skill points, what cr gear, make sure people have the appropriate mods, gen mods and white mods.

    I would prefer people used the DCUO log Analyser for their final parsing data. Its very easy to use and you can see every little detail of what your doing with this and i personally think its the best way to go. Running 30 second parses is good but using the analyser gives you better data interpretation.

    When i test i use the log analyser and use 10 second intervals and run parses for 5 min on each rotation so i have a large number to get data and averages from. If you just use a min or 2 thats not enough samples to get a real solid average. Also you want to run flawless rotations, if you mess up remember the parse dps and you can take that out of the data on the website so it wont mess up your average by having a bad rotation in there. If you mess up to many times start over. Everyone should be able to run 5 mins flawlessly its not to hard, if you can't then you don't have any place testing this stage of the powers. That will give you 30 samples to average from which the site does for you.

    Sounds like a lot of work....not really. If you want accurate testing its worth it.

    If don't use the analyser make sure you are running 30 second parses and lots of them not just 4 and call it a day. Sometimes you might just get a few lucky or unlucky ones so I like to run about 10 in a row. And same thing applies to the 30 second parses make sure you run them flawlessly, any mistakes and toss that one out. Then after you get 10 solid ones average them all out. You want to deal with averages not high and low spikes.



    I know everyones skill levels are different but it would help clean things up a bit if everyone followed the same guide lines especially in skill points, gear, white mods and also made sure they were modded properly for what they were testing.

    This is serious stuff and if we want to do this correct we can't have everyone giving different numbers cause they are using different methods for testing then someone else. Myself and a small group of others have our own standards but we are only a few people, it would be great if everyone was on the same page.


    A example of what kind of standard we could start with...

    -240 Skill points. Thats a level to 100 cr character plus 2 sp.
    -Full 175 vendor gear.
    -Correctly modded for the power, playstyle your playing with Special Forces mods
    -Correct generator mods for the power, playstyle your playing with Special Forces gen mods

    -White mods
    -Weapon: no blaster adapter, it has the chance to skew your parse a bit.
    -Neck: Escalating might/ Relentless Precision. Depending on what your playing.
    -Chests: Not sure of the math on Core vs Pen at this moment but we could just leave this mod out.
    -Hands: Max damage.

    Another thing thats going to make or brake testing is power....you need proper power to test the best possible loadouts. If you don't have 2 trolls your not getting the right results or using the best loadout with a lot of powers so your not getting the correct results.Theres absolutely no way you can test properly with just yourself at a training dummy with no power running 10 second parses.

    I think its time we give the dummies back their power return so everyone can test not just the few of us who know people and not just all in raid testing. In raid testing is great but you need a bit of both dummy and raid testing to really narrow things down. The dummies will show you whats best but the raid will show you if its actually usable in content and how it works in the real world.

    Some things work exactly the same at the dummies as they do in raid. For example most ranged loadouts will play out the same both at the dummies and in a raid but some melee builds will seem to do better at the dummies then they are in the raid because of aoe sizes and other variables. Its up to the tester to understand things like this so they can give the right feedback.

    I could go on and on about this stuff so i'll stop now lol. What I laid out here is the basics so lets make something happen so we can get this done and get it done right.


    If anyone has any questions about testing or about the DCUO log Analyser feel free to ask me.
    • Like x 6
  2. CrappyHeals Devoted Player

    For people who have never used the analyser heres one of the break down tabs that you can look at to see what things are doing. This is one of the more straight forward pages. I won't bore you with to much science stuff lol but even from this simple page you can see whats doing the majority of your damage and what percentage things are doing also the averages of each individual power which is handy.[IMG]
    • Like x 3
  3. Fatal Star 10000 Post Club

    Wait I thought we were just doing "dis power hit hard it good lololololol"?

    Crap I've been doing it wrong the whole time :oops:
    • Like x 2
  4. CrappyHeals Devoted Player

    lol when i read some peoples feedback thats what it says to me haha!
    • Like x 1
  5. spack2k Steadfast Player

    well yes but thats something for DCUO 2.0, we are 2-3 updates away from that i would say.
  6. Mighty Committed Player

    I think it'd be worthwhile to share general ranges on damage output and what's "good." I think ranges are better than out-and-out thresholds because the powers are different, and play different. Like Crappy and I talk about a lot, burst vs. DoT should have a bit of variability between them because in most cases, the burst powers are going to get up to full potential faster than the DoTs, and this'll be reflected on scoreboard damage. While scoreboard damage is BS, we all know it, we're still chasing it to some degree (albeit boss damage is more better). So if power X is a few thousand DPS behind power Y, but power Y needs a few seconds to even reach its max, whereas power X is PI+burst, then let's not immediately harrumph that power Y is OP.

    Seems like right now "good" ranged damage rotations (i.e. most powers not utilizing a melee weapon combo in a hybrid rotation) is in the 30-35K DPS area. If you're parsing in that neighborhood, with Crappy's suggested gear/SP/mod setup, you're on track.

    For melee damage rotations (very few should have straight melee setups without utilizing a weapon combo, think rage, light, etc.), "good" damage is in the 40-45k DPS area. Again, that's grown man melee, up lunging the boss, in the tank's kitchen melee, not Tunso "we're in the room together" melee.

    Guys, does this seem to jive with what you're seeing?
    • Like x 2
  7. CrappyHeals Devoted Player


    Those values are in the areas i've been getting with a lot of the playstyle. Some weapons are real messed up right now so its kinda hard to get super accurate hybrid testing. I've been testing range hybrid with mostly handblasters cause thats a combo that works well with any power so at least i have a constant variable and its easier to see discrepancies between powers. To those testing hybrid i would stick to the same weapon for each power but just remember every weapon doesn't work well it every power, thats why i stuck with hb's to test range hybrid. For melee i also stuck with the same weapon for all my testing (Ma's)

    Also people need to keep in mind when using the parser setups and burst vs dots and how those things reflect on the parser and in content. It looks like its up to us to set some standards so lets not have another round of tunso testing mistakes. And if you see something questionable don't immediately jump on the nerf or buff train, test it in all areas and discuss it with other people before committing to a powers fate.
    • Like x 3
  8. spack2k Steadfast Player

    -also i would exclude league buffs from testing since not everyone can afford them on live.

    -if there will be a standard amount of sp for testing , that amount needs to be there by default ... not everyone will have 240sp to test with for example.

    - a sp respec station in league hall would be good

    - a trinket for generating full SC bar would be usefull aswell because to gain 100% SC it cost you insane 5 min alone... ( of course this trinket is a test server only and not useable in pve/pvp instances/arena) only useable in open world if not in combat.
  9. Aerith Rose Committed Player

    Let us just make sure that all our testing is not based on 30 seconds. Problem is that powers with a longer start up will suffer more in the first 5-10 seconds and will loose a lot of add damage. I don't want that to be completely lost in the balance.
  10. McShotzz Well-Known Player

    Just started taking testing more seriously and im just wondering when you test damage parsers using the analyser are you doing it on 1/3/8 targets? Side note which is preferred for tank troll and heal between the hybrid and power mastery options?
  11. BumblingB I got better.

    1 and 3 targets. 8 targets can get out of hand, since it's possible to miss some in attacks. While 3 are the easiest to target.

    Damage splits after 2. So with 3 targets the damage would be split 200% between them.

    That of course, you can still use 8 targets, but it's possible that the numbers might get skewed.
  12. Duck X Active Player

    I think people should stay off powers and roles that they have no idea how to play, and focus on their own power and role. If you are quantum and u never played hl stay off hl and go give feedback in quantum, if you are a tank and u never dps, dont give feedback on dpsing.
    • Like x 3
  13. Surtur Well-Known Player

    It never hurts having fresh eyes look at something. It's usually a good thing because they are usually thinking outside the box.
    • Like x 1
  14. Surtur Well-Known Player

    I see people testing with max crits, overgeard, max SP and all I can do is shake my head. If you want true numbers keep your crits as close to zero as possible. Yes........... you can figure the crits in or out of the equation if you want but honestly from what I have seen of the devs they are not guuuud with numbers. I can see that too many rely on calculators and could not figure out an algebraic equation much less averages on basic math calculations without one.

    All I can say is keep your stats & CR close to the dummies you are testing. I don't trust the removal of the
    CR scaling from these guys.......... xD. Keep your stats minimal and do not use white mainframe mods because of the % boosts that they give. Whether we want to admit it or not if you are CR 200 testing on a CR 126 dummy the number's will be skewed a bit because of the stat differences between the dummy & the dummy.
  15. Duck X Active Player

    Yea thats okay but you have people that have no idea what they r doing and in their little heads think that they r doing great then give horrible feedback. If you dont know a powerset and u going to give feedback atleast have a disclamer like "hey i dont really play this power/role much but this is what i saw"
    • Like x 1
  16. Surtur Well-Known Player

    I will add that as far as powers goes... The damage out does not matter they all do not need to be 40k damage out. The devs should be able to adjust the NPC's accordingly. The goal is to balance the powers & play styles. If they all are doing 30k then so be it.........they are doing 30k. They will be balanced. All I see people doing is trying to up the numbers of everything trying balance things and that's a huge joke. more times than not you have to subtract from one thing to to get balance. It's easier and takes less man hours.
  17. light FX Steadfast Player

    Agree 100%. I made this point in another thread. Some people are literally in every single thread of every powerset. I see some who i know for a fact havent played a certain power on live but they are in that powersets testing thread giving feedback. And the response i got from my post was "people understand the issues with all these powers." And thats not accurate at all. Im not gonna name any names but some of its obvious.

    Think about this. When GM makes a new car or apple makes a new iphone do they give it to their testers and say do whatever ya want? Hell no they dont. They have em test extremes and there are parameters for testing. After the previous 2 revamps i am still surprised this is being done like this. Where anyone can go on test, do whatever they want, and post. Because nothing has changed from the previous 2 revamps as far as the process i expect this revamp to end up exactly like the previous 2 ;) Im not sure how i can even understand how anyone would believe things are going to be different this time. I dont say that to be critical either. If ya want a different result ya need to do things differently, meaning change the process.
    • Like x 1
  18. Surtur Well-Known Player

    First of all stop being a jerk.
    You automatically ***-U-ME that they are idiots by your comment ( their little heads ). Some of these guys might have a degrees in mathematics or maybe even in the top 5% of all the mathematicians in the US/world. You never know.
    Now, I do admit that there are some that give feedback that are way out there and they are easy to see. Take for example Fatals weapons thread that was started and Fatal tried to say weapons balance was ok in the first comment.
    ummmm...........OK....... LOL.

    We have devs that do not play the game and are not very good with numbers unless they have a calculator trying to fix the game.
    Then on top of that we have people giving useless feedback without numbers to back it up.
    Then we have people skewing feedback to make it so that certain play styles and powers are better than the others. This is a recipe for disaster if I have ever seen it. It's becoming a massive joke if you ask me. With so many testers either banned from feedback or just given up, the only ones left are one or two that are trying to give legit feedback for balance and then you have the others.
    • Like x 2
  19. Fatal Star 10000 Post Club

    I'm not gonna make a long detailed post, but just be short and sweet, if you guys actually knew what was going on, you would have a completely different view point. That's all I'm gonna say. Let the revamp roll as usual and let people give feedback. Everything will be ok.
    • Like x 3
  20. Mighty Committed Player


    This. We got a little sideways with each other in the main 1.7 thread, but people got some stuff off their chest, got it out of their system, and cooler heads prevailed. The dev team has never, in my experience, done as good a job as this at working with testers to find good, challenging, balanced gameplay. It's not perfect, but we're making progress every day. I have faith, man, and those who've been around since GU36 know I've never shied away from being very critical. They've earned my trust, and I hope that you'd afford them the same opportunity. They're doing good work. Hang with 'em.
    • Like x 3