I mean you get VPs for enemy units dead or fled at the end of the game right. If the character is dead at the end of the game you get his points, whether or not he joined a unit during the course of the game or not. I've never heard of somebody arguing your opponents way before. Maybe I'm misunderstanding but to me it's clear-cut.
Here's an example to show what I think your saying: a character, say a vampire lord, is in a unit of black knights. Turn 1 he dies to a cannon shot. As long as the knights survive the game he doesn't give up his VPs?!!? Of course he does: regardless of the fate of the knights.