DPS Parsers

Discussion in 'The Veterans' Lounge' started by Aelfin, May 15, 2016.

  1. Narogg Augur

    Aelfin,
    The compare feature looks AWESOME.
    Are you using a Microsoft SQL LocalDB to store the data or are you doing it all in memory with variables? Just curious, as I did not see the actual download link to the software.
  2. Deillusional Augur

    We used it for a while (the developer was in our guild, who has left the game now, but you need a windows server to collate all the logs), it worked well back then. I do not know how well it would work now.
  3. Iila Augur

    Only the healer and the healee get heal amounts reported.

    The newer versions of gamparse have heal parsing built in
    [IMG]
  4. Beimeith Lord of the Game


    Uh, what version are you using? The issue with the Y (and X) Axis labels was fixed as far as I'm aware. They used to be fixed at something like 10 or 15k intervals (for Y) but now use a logarithmic scaling to determine it. The X Axis checks the length and if they are large enough will alternate up and down so they don't bleed into each other, though this isn't perfect.

    The graphs size to however large the GamParse window is so theoretically there is no real limit to how wide you can make them. If you have a particularly lengthy fight you could stretch out the window very very wide and it would display that way. Scratch that, apparently with Windows Vista+ MS made it so you can't resize a window beyond your screen resolution. I guess I never noticed that until now.

    Well, it wouldn't be hard for me to add an option to make the graphs larger similar to how the vertical hit graphs can be extremely wide on long fights. It would cause them to have scroll bars though, and they would be quite massive depending on just how fine a scale I set them to be. This is how it was originally when I was first doing it, I had to specifically make them shrink to fit into the window because I didn't -want- them to be massively huge with scroll bars. Adding an option to choose shouldn't be hard.


    As far as spikes cramming the rest of the graphs into the bottom portion I could possibly add in an option to suppress them, but the problem is in knowing where to make the distinction. A 2 million damage/second burst might be considered a spike for one class, but it is a rather common occurrence for a Wizard at full burn with Twincast going. I could maybe, -maybe- make the value user configurable, but it is something I would have to test first.

    If I were to add an option to suppress spikes, it would be labeled on the graph in some way. I don't like the idea of having the graph be altered without it being clearly stated.
  5. Evurkvest Augur


    Cant find the files on that page. Would like to try it.
  6. Narogg Augur

    Illa, yeah I use Gamparse myself for looking at heal numbers.
    I wanted to create a Heal parser dedicated to healers since Gamparse is mostly built to look at DPS.
    I wanted to be able to pick a player and have it show the list of who healed them for each encounter and for how much. But I wanted to ONLY do it from the healers Log, which is turning out to be impossible because EQ does not log that way.

    Going to continue on with the project though, because Learning is fun!

    Right now I have the code tight enough to parse a 50 meg file in 30 seconds. not sure if that's good or bad.

    It is looking like it will have to be something the tank runs.
    Having a 105 warrior, I always wanted to know a few things.
    1. who is healing me
    2. who is the best at healing me
    3. who is over healing me

    Planning to learn how to do the graphical overlay thing so I can display that live.. wish me luck.
  7. Brogett Augur

    Aelfin, I used to use Tango! for the plots. The ability to resize them easily, to turn them into images quickly and to be able to compare any two players on the same plot was really handy. I even suggested these as feature improvements to GamParse a year or two back, but appreciate how hard it can be to do if you started off in a different direction code-wise.

    Anyway, your work was appreciated. :)

    I wouldn't use it now as I quit EQ and just periodically visit forums to reminisce.
  8. shadowgod Augur

    The best thing you can do for gamparse is to trash it and start over.
  9. Beimeith Lord of the Game


    That was kind of the plan with 2.0, but I had someone helping me (aka doing most of the work) since I don't have that kind of time, but now I don't so that's kind of on hold.
  10. Narogg Augur

    If it's written in c#. I would love to help.
  11. Iila Augur

    I want to be able to highlight a section of a graph to zoom in, and be able to load the highlighted section of the log as a fight.
    segap and Xikteny like this.
  12. Aelfin New Member

    Sorry, been a bit busy the last couple days. I'll try to comment on a few things.

    Beimeith: Yar, it was Gama's first foray. And truth to tell, a successful one, warts and all. Is Tango coded better? Sure. Much better architecture and performance. Could it be better? You bet! Side projects tend to stay in "prototype" mode as opposed to stuff I get paid money for at work ;) As I look through its code I'm certainly noticing lots of places it could use improvement. Some of that is time (Tango is .Net 2.0 and lacks... lots of advancements), some is technique (IoC, AoP, linq, lambdas, etc) and some is just professional growth. I was pretty good 11 years ago when the first version of Tango was released upon the poor, unwitting souls of The Druid's Grove and I'm better now.

    Narogg: well, sure, ultimately most text file reading comes down to a Stream of some sort. FWIW, you can simplify with:
    Code:
    while((line = sr.ReadLine()) != null) {
        // parse line
    }
    Also, no to local db (wasn't around in 2005-ish). How big is a file? 50mb? 100mb? Pittance, really, even back then. Tango does use compression for storage tho, so I can suck in an entire file and reference lines while viewing. Beyond that, 100mb of text file translates to even smaller data structures. Quick test shows that after parsing 160mb of data Tango is only using 120mb of memory so... not too bad.

    Finally, 30s for 50mb *sounds* kinda slow. For reference, Tango can rip through 100mb in ~10s.
    Code:
    ParseFile complete. [time:00:00:09.8749858; lines:990,584; l/s:100,312.45; size:80.58mb; mb/s:8.16]
    In the end tho... does it matter? If your project is a) fun b) helping you learn and c) satisfying the requirements then does it matter if it's 30s?

    Kamea: I was lucky enough to find a good graphing component at the time. In Tango you could just pan & zoom all ya wanted in the graphs

    Brogett: /wave! I think you were the impetus for DI in Tango if I remember right ;)

    Beyond that, spent a bit more time going through my old web site. Like a time capsule. Found some other images of Tango 0.5.2:
    http://www.personaluse.org/tango/tpre01.jpg
    http://www.personaluse.org/tango/tpre02.jpg
    http://www.personaluse.org/tango/example01.jpg
    http://www.personaluse.org/tango/example02.jpg
    http://www.personaluse.org/tango/example03.jpg
    http://www.personaluse.org/tango/example04.jpg
    http://www.personaluse.org/tango/example05.jpg
    http://www.personaluse.org/tango/example06.jpg
    http://www.personaluse.org/tango/example07.jpg
    http://www.personaluse.org/tango/example08.jpg
    http://www.personaluse.org/tango/example09.jpg
    http://www.personaluse.org/tango/example10.jpg
    http://www.personaluse.org/tango/example11.jpg

    html output
    http://www.personaluse.org/tango/Veldyns_Shade_2007_11_9.htm
    http://www.personaluse.org/tango/vs.html

    visual cues
    http://www.personaluse.org/tango/ontoptest.jpg
    http://www.personaluse.org/tango/visualcuetest01.jpg
    http://www.personaluse.org/tango/visualcuetest02.jpg
    http://www.personaluse.org/tango/triggerseditor.jpg

    And then I found an interesting directory hidden away that I had totally forgotten about! I had completely forgotten that I did a ground-up rewrite in 2008-2009. I found the source to v0.6 and a few screen shots of new stuff. Mostly around healing and tanking stats.

    new layout with stats:
    http://www.personaluse.org/tango/veldynexample.png
    http://www.personaluse.org/tango/veldynexampleod.png

    graphs will spells
    http://www.personaluse.org/tango/irekkvs.jpg

    http://www.personaluse.org/tango/merctanktest01.png
    http://www.personaluse.org/tango/merctest02dispread.png
    http://www.personaluse.org/tango/merctest02def.png
    http://www.personaluse.org/tango/merctest02off.png
    http://www.personaluse.org/tango/merctest02offgraph.png

    example of new graphs and stats, tanks:
    http://www.personaluse.org/tango/jatarat.png
    http://www.personaluse.org/tango/jatarats.png
    http://www.personaluse.org/tango/jataratsbs.png
    http://www.personaluse.org/tango/jatardts.png
    http://www.personaluse.org/tango/jatardtsbs.png
    http://www.personaluse.org/tango/jatarspells.png
    http://www.personaluse.org/tango/jatarstats.png
    http://www.personaluse.org/tango/tearsinat.png
    http://www.personaluse.org/tango/tearsinats.png
    http://www.personaluse.org/tango/tearsinatsbs.png
    http://www.personaluse.org/tango/tearsindts.png
    http://www.personaluse.org/tango/tearsindtsbs.png
    http://www.personaluse.org/tango/tearsinspells.png
    http://www.personaluse.org/tango/tearsinstats.png

    more stuff:
    http://www.personaluse.org/tango/patch01.jpg
    http://www.personaluse.org/tango/patch02.jpg
    http://www.personaluse.org/tango/patch03.jpg
    http://www.personaluse.org/tango/lethar1.jpg
    http://www.personaluse.org/tango/letharramp.jpg
    http://www.personaluse.org/tango/letharspells.jpg
    http://www.personaluse.org/tango/selayspells.jpg

    Anyway, I did manage to get v0.6 to compile but it took a bit of massaging. ILMerge is having fits with it atm, pretty certain that is because it's in a funky state of having some 2.0 and 4.0 framework pieces.

    The other problem I ran into is... I have no EQ logs to test! HAHAHAHA... I can't believe I have lost all the test files I used to use but hell that dev computer was probably 4 or 5 machines back in time. Errr... anyone have some large, zipped up log files I could use to see what happens?
    Sirene_Fippy likes this.
  13. iZealot Lorekeeper

    Gamparse has been really lagging me lately all the sudden, as of 4-6 weeks ago. Maybe a patch or something? I can solve it by turning "other hits other" off, but i'd rather not do that. All on an SSD.
  14. Xikteny Augur

    Is GP lagging or does the game just lag itself when you don't filter out other people's hits?

    Many people do seem to have lag problems when logging the entirety of a raid. To throw out a totally random guess: if you use any sort of antivirus software (even the ones which come with Windows), adding an exclusion such that said software doesn't monitor EQ's log directory is often helpful.
  15. -wycca Augur

    Wow Aelf - I loved Tango back in the day. I actually tried to hunt you down a few months ago across various forums you posted Tango info at to try to get an old copy (not even sure if it would still work).

    Crazy that you're back around =)
  16. Gundolin Augur

    Ok, not wanting to open a new thread for a dumb question and it is vaguely related to the topic so.

    I am not capturing other people's melee dps in my parse. It is not reaching my filter in my spam window and I can't figure out why. I have tried toggling it on and off several times in the filter settings. Is there something basic I am missing or do you have to be almost right on top of a person to see their melee dps?

    Thanks in advance.
  17. Narogg Augur

    Options
    Chat
    Select category - Melee
    other damage other - show
    Gundolin likes this.
  18. iZealot Lorekeeper


    It's EQ only. it's not the virus prot either, i've turned disabling real time protection, etc and excluding that entire hard drive EQ is on.

    Perhaps my vid card or cpu is crapping out.
  19. Narogg Augur

    I am having trouble with encounter detection in my heal parser.
    I was basing the encounter name on parse on "hits" since I believe every mob "hits"
    Example log text
    [Tue Feb 09 22:43:32 2016] High Keeper Arieal hits Vegettomota for 9900 points of damage.
    This works fine for 99 percent of the encounter detections in 99 percent of my log samples.


    My issue is, it is picking up some players/pets as the encounter instead of the mob name.
    Example log text
    [Tue Feb 09 22:43:27 2016] Lebobtik hits a wildfire phoenix for 1416 points of damage.
    [Tue Feb 09 22:43:48 2016] Elmindor hits a wildfire phoenix for 13559 points of damage.

    @Roshen
    So my question is the game reporting these lines as "Hits" when they should be "hit"?

    To my knowledge it is only rangers, berserkers that are logging this way.

    The un-named pets are easy to filter out but the Necro player named pets are also logging this way.
  20. Dropfast Augur

    For anyone who is having issues with lag while parsing the hole raid or even just yourself, try /loginterval 1. About 6 months ago or so, it seems /loginterval 0 got broke for many people. This will not solve the lag issue for everyone but I know it's helped a lot of people including myself. I've never had any lag issues at all parsing the hole raid with no filters on until that patch a while ago. I went from 0 lag to insane lag out of no where. So I tried /loginterval 1 for the heck of it and bam, back to 0 lag parsing the hole raid.(No SSD either)

    The down side to this is your AT's from Gina will now be 1 second behind. So if you react to AT's slow this will just make it worse but for many it's better than lagging your butt off all the time. If you are really slow at reacting to AT's, then I do not recommend this.