Only darklights see stealthed infiltrators. HV/NV scopes don't pick up on stealth. Otherwise, who would in their right mind use a limited cone Darklight if they can make an infiltrator show up all over their screen at longer ranges? And infiltrators and LA have a lot of common ground. Both their abilities allow them to infiltrate bases and ambush people. The LA can get to high ground much more easily to spot people than the infiltrator, and he's usually in a better position to spot important targets rather than the targets everyone can already see. The way these goggles spot also improves the ambush and tracking ability, allowing the LA to be a better hunter of his prey. Infiltrators use darts and motions spotters to detect for the entire team, these goggles would benefit the LA only, and only for the people he spots. Otherwise you might as well say "the Engineer is the explosives class, so only he is able to carry any C4 or mines. No more AI mines for infiltrator or C4 for Heavy/Medic/LA", it's the same reasoning.