Truebox code causing disconnects Issue Ticket Created on DBG issue tracker

Discussion in 'Player Support' started by Risiko, Nov 23, 2016.

  1. Machen New Member



    To quote you from the other thread:

    Pikallo likes this.
  2. Astley Augur

    Quick question Illusory...

    What's your connection setup like? Are they all wired into the same router? Or connecting wirelessly to either the router, or to a wi-fi enabled modem?
  3. Machen New Member


    Ok, I had a couple krono left over from before I quit so i figured, why not try it myself.

    Setup - two computers, one my standard home desktop, running very little in the background (no virus software etc.) The other, a surface pro 2, running nothing other than what came with Windows and the EQ client. Two toons, in a group, each spamming the 2 hotkey (using my middle fingers for what should be obvious reasons.) Zero third party software running.

    Test 1 - both computers connected on my home network, connected over wifi to a wifi router which connects directly to a cable modem and then to my isp (not AT&T). Very simple and standard setup to both the wifi router and the cable modem. 14 seconds of pounding both 2 keys as quickly as possible disconnected me.

    Test 2 - both computers connected over wifi on my AT&T cell phone hot spot. Nothing running through my home network (I unplugged the cable modem to make extra sure.) 6 seconds of pounding both 2 keys as quickly as possible disconnected me.

    Test 3 - one computer connected to home network, one connected to cell phone network. After two minutes of pounding both keys my arms grew tired and I stopped with no disconnect.

    I did a tracert on both connections, and it looks like the first hop they have in common is a backbone router in Las Vegas (>1000 miles from here.) After that both connections followed the same path.

    My personal conclusion from this: If it is a network thing, it's either something related to the internet itself (in my case something in between Las Vegas and the EQ servers), or it's something so common that I had exactly the same issue using two different ISP's (and therefore a lot of other people would have the problem too.) I'm not a network engineer though, so if someone who is has another theory I'd love to hear it. It would probably be good to repeat these tests for more robust data, but I'm not gonna do that when it still seems a possibility it might flag me for stricter enforcement.
  4. Accipiter Old Timer


    Good info. Thanks for doing this.

    It's probably coincidental that I'm in Las Vegas but I can't help but notice. :)
  5. Astley Augur


    You never know ;)

    If Risiko is routing through Las Vegas also, that would be sort of funny I think...
  6. Machen New Member


    From what I recall about half the country probably routes through Las Vegas on their way to the Everquest servers.
  7. Astley Augur

    Tested a couple of things.

    1) Through my connection, I could trigger it like Risiko was before by repeatedly spamming key presses on 2 characters in the same zone. (Nexus)

    2) Then I attempted to mimic this action after zoning one character into Netherbian and leaving the other in the Nexus. Repeatedly spamming my hotkeys had zero effect. No disconnect even though I actually ramped up faster than the previous test.

    3) Moved characters to opposite ends of the long hallway in Netherbian Lair... Began to ramp up speed again, took about 30 seconds to trigger a dual disconnect based on about 1-2 key presses per second.


    So, it's not just close proximity within the same zone... if they are within the same zone at all, repeated key presses seems that it can trigger this.

    Next tests

    4) Logged characters back in and opened up /tell messages to each other. Started typing random letters, spaces, numbers repeatedly as fast as I could... No disconnect.

    5) Started new /tell and started spamming same number presses that disconnected me in 3) above. No disconnect.

    6) Closed /tells and started hitting same hotkeys from 3) again, disconnected quickly 10-20 seconds.

    7) Logged back in and started spamming hotkey 2 repeatedly on one computer and hotkey 3 repeatedly on second computer. No disconnect noticed for at least 30 seconds.

    8) Continued spamming hotkey 2 on one computer and started alternating hotkeys 2 & 3 in quick succession, changing randomly and for longer and longer series of pressing 2 simultanteously. No disconnect noticed for at least 30 seconds.

    9) Rotated back to spamming hotkey 2 on both computers. Disconnect within 15 seconds.

    10) Started spamming 'D' to simultaneously rotate characters. No disconnect noticed for at least 30 seconds.

    11) Switched to spamming different hotkeys 7 on one computer and 8 on other (8 had no active ability tied to it) Disconnect within 15 seconds.




    So, from some brief analysis, it seems primarily triggered by being in the same zone and hitting the same hotkey. It was not caused by typing messages, was not caused by random, quick key presses... Only seemed to be caused by hitting the same hotkey repeatedly, not other, non-hotkey commands. The buttons that triggered the disconnect would not trigger during a /tell either.... So it does seem to be hotkey command press submission connected.


    So, best I can suggest, is that it's something to do with whatever commands you're sending out in succession. Either they need to be varied more, or slowed down, or something else is communicating to Everquest that you're firing the same key press repeatedly. I don't know, maybe you've got a slightly sticky keyboard and it's multi-firing key commands without you knowing?.

    Anyway ruled out a few issues there to see what I could do to trigger the alerts and seemed to narrow it down to JUST hotkey press relation.

    Next step is to play for awhile back in my normal method and see if anything starts causing me to disconnect suddenly.
  8. Machen New Member


    For most of us, that's the case. But look at the video Accipiter posted. He's not madly pounding away at the keys--it looks like he's hitting about 1 keystroke per second which is pretty tame for a boxer. And he's not even touching a second keyboard at all on one of his disconnects.

    In my situation, I can reproduce disconnecting by spamming keys. But my wife and I have boxed together for months without accidentally triggering it (with each of us playing 2-3 characters during most of classic/kunark on Phinny.) This suggests that the threshold for her and I is very different (for whatever reason) than the threshold for Risiko and his roommate. Maybe there's something in the internet setup that is causing that difference, after thinking it through some more I think that's still a plausible theory--because the test I ran clearly didn't hit the bug or whatever that Risiko/Accipiter experience, and neither did yours. As far as I can tell, what you and I both did tonight showed that for both of us, it is working as intended. We have to really work at it to get those disconnects. So maybe our connections are "normal" and theirs are screwed up somehow.
  9. Astley Augur

    Right Machen...

    What I'm thinking, is possibly... in Accipiter's case the amount of hotkey spam needed to trigger a disconnect is multiplied by the number of computers connected.

    So, if 2 boxes are hooked up at the same time, maybe simultaneous key presses every half-second.

    If 3 boxes - 1 every 2 seconds

    If 4 boxes - 1 every 4 seconds


    Or something like that.


    And in Risiko's case, it may be they're button mashing faster than needed without realizing it.
  10. Risiko Augur

    Thanks for the help guys. It's greatly appreciated, and 100% more help than we are getting from DBG /sigh.

    By the way, I am in a suburb of Dallas, Texas. I don't know for sure if I route through Las Vegas, but I would think it is highly likely as Las Vegas is a major hub between central US and west coast US.

    One thing about the way the issue triggers for my brother and I that I should point out is that we don't normally spam keys. I spammed keys in the video because I was wanting to get the issue to trigger quickly for the video. After watching Accipiter's video where he did it normally and just let it happen naturally, I wish I would have done my video that way. His video shows the issue in it's natural state.

    Also, in my video, I was only using my desktop computer and my notebook computer. This is not how my brother and I play. Normally, I play on my desktop in my room, and my brother plays on his notebook computer in his room. To show both computers on the video for demonstration, I used my desktop computer and my notebook computer.

    During a normal play session for my brother and I, the facts are as follows:
    • I am in my room on my desktop computer
    • My brother is in his room on his notebook computer
    • Both computers are connected directly to the router in my room
    • Router is connected directly to the FiOS modem
    During a normal play session for my brother and I, the scenario and how it plays out:
    1. We group up
    2. Go to a zone
    3. One of us will pull a mob with his pet (mage), his nuke, or my nuke
    4. Casting normally (nuke, root, or pet heal). No spamming
    5. The first time usually takes a while before it triggers a disconnect
    6. The disconnect will come on one of us casting normally. Again no spamming. You only hit nuke once, and then you wait 5 seconds for it to finish casting, so it's not like you are spamming keys
    7. After it disconnects us the first time, it seems to get faster at doing the disconnects after we log back in
    8. The disconnects are immediate... no lag... no rubber banding... just instant quick to disconnect screen
    9. Logging back in can be done immediately. There is no "you have a character in game" message
    What used to happen before the last patch:
    1. We group up
    2. Go to a zone
    3. One of will pull a mob with his pet (mage), his nuke, or my nuke
    4. Casting normally (nuke, root, or pet heal). No spamming
    5. The disconnect would usually take a bit before it happened
    6. The disconnect would come on one us casting normally. No spamming
    7. The disconnect would be immediate, no lag, no rubber banding, just instant quick to disconnect screen
    8. Logging back in would be done immediately. There is no "you have a character in game" message
    9. *IMPORTANT* ... After the issue would happen once, it would not happen again in that same zone for the duration of our play session
    10. *IMPORTANT* ... If we moved to another zone, and started killing, the issue would happen once for that zone for the duration of our play session. It was always 1 disconnect issue per zone per play session.
    The thing to note here is that everything is the same from before the last patch and after the last patch except for one thing. Before the last patch, the issue would happen ONLY ONCE per zone per play session. After the last patch, the issue happens repeatedly per play session. This leads me to believe they changed their code to be more aggressive in the last patch.
    I hope all of this information helps.
  11. Astley Augur

    Risiko...

    I have a feeling that the real issue with the disconnects is going to be something with the type of key presses and the way those inputs are being filtered into your router and sent to the server.

    I'm not sure what your setup and gameplay routine is... but if it's similar to what I see in grouping myself with casters, you're set up in a spot, and casting, without moving much from your sit spot.

    If you're just hitting hotkeys to trigger spells and sit/stand rotations... I would suggest trying something else during your next play session with your brother. In the midst of your play sessions, add in some slight character turns in between hotkey presses.

    If your hotkeys are set up for something to like... stand, cast spell gem 3, sit... and you rinse/repeat that, I suspect this could be an underlying issue. It was primarily the hotkey triggering that I found to be the issue.

    Remove some of the automation from your routine to hotkey pressing. Mix in a directional turn between casts. Spamming the 'D' button to make slight rotations on my characters didn't set off the true-box alert. So, perhaps if you and your brother manually shift your characters between casts, adding that in... it will break up the hotkey triggering alerts enough to 'reset' the trigger that is disconnecting you guys.
  12. Accipiter Old Timer


    Maybe but as you can see I don't hit buttons particularly fast. My typical start-of-fight looks like this:

    SK: 2
    Bard: = -
    Shaman: = 1 9
    Cleric: = 3 or maybe F2 5 depending on SK health
    Wizard: 1 7
    SK: 4
    Wizard: 2
    Shaman: 6 (maybe)
    SK: 4
    Wizard: 3 or 6

    So my assist key (=) on 3 computers is the same but otherwise there isn't a lot in common.

    I am now thinking my situation has little to do with keys. It's something in my network that triggers the true-box code. Could it somehow be that packets get queued and sent in a bunch and it looks to the server like multiple key presses? I don't know enough about how routers and modems work to know if that's possible.
  13. Machen New Member


    It's possible but I still think this is unlikely. First, if that's how they did it, it would be pretty terrible design -- because latency on the internet would make the packets arrive at different times anyway in many cases, and because in other cases because of latency, packets that were sent somewhat apart in time would randomly arrive together. I think if that's how they did it, we'd see a lot more boxers randomly disconnecting at times. It's far more likely they use the timestamp from when the packet was sent than when it was received. But who knows, maybe that's the design flaw.
  14. Risiko Augur

    Logic would argue that it can't be an issue related to anything on the player's side because:
    1. It only happens on the Phinigel server
    2. It only happens when your 2 or more accounts' characters are in the same zone together
    3. It only happens when your 2 or more keyboards hit a key at the same time
    4. It does not happen when you are sending a tell
    5. After it happens the first time, it happens much faster on subsequent logins for that play session
    As a programmer by profession for the past two decades this is how I see it.
    If it was client-side related the following would be true:
    1. It would happen on all everquest servers
    2. It could happen even when you are sending a tell
    3. It would happen at the same rate no matter if it was the first time it happened for the play session or the 100th time
    If it was network related the following would be true:
    1. It could happen on any everquest server
    2. It could happen no matter if your characters are in the same zone or not
    3. It could happen no matter if two keys were hit at the same time or not
    4. It could happen even when you are sending a tell
    5. It would happen at the same rate no matter if it was the first time it happened for the play session or the 100th time
    This indicates that the issue is a controlled reaction pattern that is based on a rule set being applied at some level on the server. I can rule out the possibility of this being a rule set being applied at the client level because of the fact that the issue logs out the client from the server immediately and does not leave the character logged in to the server.
    If the client was disconnecting the character, the server would still show the character as logged in to the server for some time as well as leave the character vulnerable to attack by mobs during that time which it does not. When this issue happens, your character is immediately logged out even if they are in mid battle.

    That's why with out being a programmer at DBG, I can honestly say that this is something that is coded in to the system. The pattern does not fit any other possibility.




    Granted I didn't go in to minute detail on "casting nuke" in my scenarios above, that didn't mean that I use hot key macros to "stand, cast, sit".

    What I meant is that my brother and I run our characters around, click on a target or assist or tab target or what ever. Then we do something which usually involves pressing a key to casts a spell. That key literally would be just the spell icon dragged to the hot bar for that key. It's not a macro.

    There literally is no automation.
  15. Risiko Augur

    I have to agree with you that it doesn't sound likely.

    One thing that just occurred to me after reading what you and Accipiter said, is that this could be a client-side or network triggered and server-side implemented.

    What I mean is that it's possible that this is client-side only so much so that it is being triggered based off of something client-side or network, but it is clear to me that the actual implementation of the disconnect affect is server-side meaning there is code on the server that interprets what it believes to be multi-box activity and initiates a disconnect.

    I say that it is the server side that is initiating the disconnect because there is set of rules being applied based off of our tests and individual experiences with the issue.

    Again, the rule-set being applied includes:
    1. Are you on the Phinigel server
    2. Are your accounts logged in on the same IP address
    3. Are your accounts' characters currently logged in on in the same zone
    4. Are your accounts' characters implementing an ability (not casting a spell is an ability of a spell)
    For anybody that might be like, "How do you know what rule-set might being applied to the situation?" It's a lot like how we know that there is a rule-set in the universe for Physics. Nobody spoke to God, and asked, "What are the rules for Physics?" No. Instead, scientists came up with hypothesis, tested those hypthesis with tests, and came up with what we today call the laws of Physics.

    We have created hypothesis, tested them, and came up with a list of rules that are being applied to the disconnect issue. We don't have to talk to DBG employees to figure that a rule-set is being applied.
    Machen likes this.
  16. Machen New Member


    I think it's possible but I'd be surprised. Shouldn't put anti cheat software client side when they can put it server side. Given the history of m q and what it does, it would be pretty silly to put the trigger client side when people have the know how to simply modify the client to block it.
  17. Astley Augur

    Risiko... I think you're missing the point.
    Yes, it's the true-box code related. But the code is being triggered by what is going on on your computer. Whatever algorithm they have to determine that is likely looking for some sort of specific action being taken between multiple computers on the same network.
    In Accipiter's video, he wasn't moving around, and neither were you in yours.

    The code was implemented to fight bot-armies that were sending automatic hotkey presses too many times within a set period of time.

    So, most likely... you need to vary what you and your brother are doing between hotkey presses when in the same zone.



    That's automation... that's the hotkey shortcuts. If you dragged your "SIT" ability there or a Clicky Effect from a piece of armor...

    What I'm saying is that it could be player-side... because you are controlling your character primarily through hitting HOTKEYS instead of say, clicking with mouse, and breaking up the monotony of hotkey presses with moving around in position, etc.

    And by player-side I don't mean it's some fault or error... I'm saying the input you are sending... specifically a series of hotkey presses without some motion or randomized break up to the routine... emulates some part of what the true-box code is looking for in the algorithm to disconnect bot-armies.

    Let's say, the true-box code has a trigger that is similar to this: "If 50 hotkey presses are made by 2 or more computers without being broken up by a mouse-click, or keyboard command, then DISCONNECT"


    Look at Accipiter's entry list above. His routine when he is boxxing his group is ALL hotkey controlled.


    If the 2 of you tried controlling your characters with more physical movement, and/or mouse-clicking instead of keyboard entry, then I'm thinking you would stop triggering whatever rule of the true-box code it thinks makes you look like you're running an automation program.

    My feeling is it's not programmed specifically for simultaneous key-presses... I'm thinking the code has a set limit to like, hotkey-presses per minute without interruption.

    So, the fix, is more manual control than hotkey control.
  18. Machen New Member


    You're still missing that some of us can hit way more hotkeys per minute than either of these guys do and not ever disconnect.

    Also, accipiter d/c once in the video when the only thing he was doing was moving (on one character) -- using movement keys not mouse, but still. Possibly there's a delay on it and he was actually d/c for what he did a few seconds earlier, but I dunno.
  19. Astley Augur


    Nope. I'm not missing that. As I said earlier, it may be increased by number of boxes connected at once. Like, with 2 computers on same network, each press may be worth X% of the limit. With 3 units, each press may be worth 3X%, 4 becomes 4X%, 5 becomes 5X%... etc....

    This means that Accipiter, by having more than a few, reaches the limit faster than you or I would Machen.

    And every disconnect would have a bit of delay to it built into the system likely. You wouldn't want a mass-botter to figure out exactly what key press rotation it was that got them booted immediately.
  20. ChangedName Lorekeeper

    It doesn't even have to be the same hot key. This is really annoying. Maybe DBG should stop trying to play 'police' and start improving their games.