DPS Parsers

Discussion in 'The Veterans' Lounge' started by Aelfin, May 15, 2016.

  1. Aelfin New Member

    Howdy!

    Curious, how goes dps parsing in the EQ world lately?

    I used to write a parser called Tango! many years ago, stopped when my daughter was born.

    Starting to get the itch again ;) Been doing a lot of mid/back-end code for web services for quite a while, doing a front-end windows app as a personal project could be a bit of fun. Maybe open source it this time and throw it up on git hub.

    Gotta admit, not played EQ in ages so not too sure how lively the community is.
  2. Heile Lorekeeper

  3. Raptour_MT Elder

    Unfortunately Gamparse is it as flawed as it is with the range issues on a spread out raid. I would prefer a web based parser that you log into that could monitor each person in the raid individually, something similar to TorParse or Parsec from SWTOR, would be more accurate with out any range issues, and could track more info from healers and support.
  4. Sancus Augur

    It's a miracle that Gamparse is able to give the information it does given how awful and arcane EQ's damage reporting and logging system is. Uploading logs to a website to get a more accurate parse would be cool, but it's a *much* more massive undertaking than a parser, if it's even possible given all the discrepancies between two EQ logs of the same fight.
  5. Xikteny Augur

  6. Aelfin New Member

    Sounds like Gamparse is handling most peeps parsing needs?

    No problem. I had just revisited some of Tango's code the other day to show a work mate that 550 mb/s parsing is no problem in C# and it just got me curious, thought an update of some sort might be a fun side thing ;)
  7. Beimeith Lord of the Game

    I took over development of GamParse after Gam got too busy to do much himself. If you're interested I could possibly use some help with maintaining/updating it.
  8. Aelfin New Member

    Ahhhh... Thanks Beimeith. I don't mean this badly, I really don't, because I remember having chats with Gama years ago. Gamparse was his first project and I remember looking at some of that code and just... no. It was his first code project I believe and that kind of.... organic... code spread is pretty common.

    I just downloaded 1.5.1.7 and took a quick look at it. That MainScreen.cs form is monolithic with something like 15K lines itself. I don't envy you dude, I've no interest in digging into that. Major props to you for taking that on and maintaining and enhancing.
  9. Draig Journeyman

    WTB combined logs. Client/Server model with only self being reported to the server for real time merging and parsing seems to be the best method to go about it without some really funky time corrections and duplicate removals involved. It's been done before and by all accounts was pretty amazing, but the author is scarce and never released the code or the server, only the client - so it's pretty much dead now.

    If you release something, do it open source please so it can live on long after you have given up on it.
  10. Beimeith Lord of the Game


    There's a reason why I call it "The parser that EQ deserves." Spaghetti coded game, spaghetti coded parser. Match made in hell. :)

    I only just recently finished renaming everything from Label23, TextBox76, etc. to descriptive names lol.
    adetia and Xianzu_Monk_Tunare like this.
  11. Narogg Augur

    I wrote a C # log parser I was intending to use to parse healing in EQ... then after all the code was written and hours of my life were given, it occurred to me that the actual heal data is outputted to the recipients log and not the caster.
    // Logic Error

    Until the Dev's move the healed amount data to the casters logs, we will never have a stand alone heal parser.

    The idea of having a joint web based parser is pretty neat, but if you have 54 people all uploading the same 1 line of text with different time stamps due to lag, you would have to sort thru 54 regex's per uploaded line, discarding duplicates, and comparing any variations in the amounts and building a master log to actually parse from.

    Neat concept, but due to the resources needed to parse that much data remotely, I think most hosting companies would say your over utilizing the Virtual CPU and Memory and ask you to stop or they will throttle your site.

    As a student of programming, I would be most interested to see how Gam decided to parse the file vs what I am doing or I can send you my parser .cs file for a peek and see if I am on the right track.
  12. Beimeith Lord of the Game

    Without meaning to insult Gam, it parses rather badly to be honest. Well, I take that back. It does work fairly well, but it could definitely be improved in many ways yet. In his defense though, he wrote it as a school project when he was learning to program if I remember correctly.

    My first EQ related project I started for school was in VB.net and was an "Enhanced AA window." Basically it worked just like the AA window in EQ, but with additional descriptions of what the AAs did. You could use it to plan which AAs you were going to buy (it would tell you the cost for example) or see more information about the AAs, etc. I looked at the code recently and it was just god awful. It worked, but it was awful.

    As an example for GamParse and the...unique choices Gam made, lots of things were/are driven by running timers instead of being event driven.

    As a specific example: Control over savings the settings file used to be done by a timer. Whenever you changed a setting that was saved to the settings.ini file, it would set a variable savesettings = TRUE. A timer (timer4) would check every second if savesettings = TRUE and if it was it would trigger savesettings() which would then overwrite the settings.ini file with the current values and then reset savesettings = FALSE.

    I changed all that. I took out timer4 and the savesettings variable and made it all so that settings are now saved by clicking on the "save" button on the specific option forms (or when you select a backup folder). This directly calls savesettings() and writes the new settings.ini values.

    There is no more timer checking every second, no overwriting your settings on accident because you changed the wrong thing, etc.

    There was another timer that controlled something that was completely pointless. Timer 2 started when you loaded a log file and basically triggered part of the load separately. As in, the load process would start, trigger the timer, the timer would tick once, complete the load, then the timer would turn off and never be used again unless you loaded a new file. It was entirely unnecessary, lol. I took out timer2 and moved all the code that was triggered by timer2 into the main load.

    I asked Gam about that one and I believe he said he did it because he was experimenting with timers and how they work and that's pretty much it.

    The main loading of data is still controlled by a timer. I believe that one runs every 500ms and pulls in x amount of data that's been written to the file. It was on the list of things to change, but that ones a little more complex so it hasn't been done yet.


    So yeah, thats some of the internal workings.
    Xianzu_Monk_Tunare and Nniki like this.
  13. Narogg Augur

    This might not be top notch programming, but it is how I am doing it. I am not parsing a " active file" at the moment though, I am making a working copy of it then parsing the working copy line by line. The concept should still be the same though.
    /// Stream Read the TXT File line by line

    using (StreamReader reader = new StreamReader("Working_file.Txt"))
    while (reader.EndOfStream == false)
    {
    string line = reader.ReadLine();
    if (line == null)
    break;
  14. Aelfin New Member

    hadn't visited my own site in years. got its gears spinning again. dumped all the forum stuff but did find some of the docs

    http://www.personaluse.org/tango/help/index.html

    is this even up to par anymore? of any use? i could look at open sourcing the last 0.5.2 version. do realize it is pretty old at this point, .net 2.0 code.
  15. Kamea Augur

    One problem with most parsers is a lack of graphing options.

    Specifically, being able to adjust what's displayed on the Y axis to nullify spike damage, and scale the X axis so you look closely at certain time frames on long fights, would be nice.

    GamParse's graphed get quite bugged (unreadable text for Y axis labels) if I have a big rampage round. And even if the text is readable, spike damage will often leave the rest of the fight on the bottom ~15% of the graph window. On long fights, it can be hard to read into tank graphs if the damage / heal lines aren't spaced out enough.
  16. Iila Augur

    The healer does get the total amount healed.



    You have healed Loperz for 18834 points. <--Actual amount healed
    Loperz's skin glows with lunar energy.
    You have healed Illa for 41401 points. <--Actual amount healed
    Your skin tingles with lunar energy. You have been healed for 41401 points. <-- Amount attempted to heal


    Myzr has healed you for 9615 points. <- Actual amount healed.
    You experience a healing surge. You have been healed for 18564 points. <-- Amount attempted to heal

    "You have healed <name> for X points." is the real amount healed, as reported to the healer.
    "You have been healed for X points" is the amount that spell attempted to heal, and is only reported to the character receiving the heal.
    "<name> has healed you for X points." is the real amount healed on you by another character.
  17. Narogg Augur

    Let me make sure I understand what your saying Illa.
    For the tanking tab

    Caster has healed you for X points is how much it took to get you to 100 percent.
    You have been healed for Y points is how much the spell could have done if you were not topped off to 100 percent.

    So we should actually have a over heal stat on the tanking tab where X is subtracted from Y.

    On the healer tab
    "You have healed <name> for X points." is the only thing that needs parsing?
  18. Narogg Augur

    I was looking at the "You have been healed for X points" and thinking that the healer was not getting all the data.

    This information changes things and I think I can now spend my day off working on my heal parser.

    In theory,
    "You have healed <name> for X points." and "<name> has healed you for X points." should always be equal?
  19. Xianzu_Monk_Tunare Augur

    The fault of the spread out raid is not with Gamparse but with the games logging system itself.
    Sancus likes this.
  20. Narogg Augur

    Illa, just ran into a snag,
    I can not see any of the other healer's heals in my healers log.
    So the parser only works if you load the tanks log file.
    This is the same issue I created a ticket for back when I was feverishly working on this project.
    https://dgcissuetracker.com/browse/EQ-50