IIRC it is determined by random if the besieging or the besieged army gets chosen. Scales affect the one who owns the province, that would be the sieging army.
If the besieging army's nation casts the spell, it hits the guys in the fort unless it results in attack by special monsters.
If defender uses the spell, it hits the besiegers unless result is attack by special monsters. This is due to not being able to target one's own troops (unless the spell is not interested in the identity of targets).
If it's cast by a third party, it's more totally random.
At least, this is the impression I've gleaned from several threads. I could be wrong, though.