Gem use question.

Those Castles could have not been repaired from their Last siege, had bonuses to siegeing in some way or otherwise been able to batter down whatever fortress you had on their own. This is remarkably true if you've been fighting long drawn out wars on multiple fronts and your castles are in a constant state of disrepair.