Vortex VM21 Charging resets to 1 and stops when you toggle between Zealot Overdrive On and Off. Unless you are shooting with it Off the whole time until shooting or On the whole time until Shooting, the Charge resets to 1 bar and stays that way : (
I always considered this to be by design to prevent people from flicking it on just before firing and then turning it off again after, Which would allow you to deal your bonus damage without taking increased incoming damage. But honestly it's such a small boost that I wouldn't mind if they let you. ZOE just needs a total rework.
