That's how it works during regular combat, but at one point, years ago, I found myself standing next to a KOS mob which was fighting another group's pet. For some reason which I don't remember any more I couldn't easily move out of the way, so I fully expected the mob to turn on me. However, it continued fighting the pet, presumably because I wasn't on its hate list even though I was KOS. I am not sure if it still works that way or, if it does, if the behavior is consistent.
If there are mobs that only attack specially (think Octa only engaged those in the raid that attacked it, or in TBS those portal spawn mobs were changed by Devs to only aggro the one who triggered it), then yeah that probably is what you experienced. Root also plays havoc with aggro on pet tanking, but that's not related to anything you've said/experienced.
The damage scales up as you level, and should you complete a level of Slayer accomplishments, you get a faster cool down time ... I have like 8 more mob types to go to get the first of that on my main ...
