[Excist]

The way I think of it is that each 'swing' isn't really a swing, but more of a measure of what they accomplished in a set amount of time based on action points and their capabilities.

As such the swinging of the weapon is a minor part of their activity and has less effect on how quickly they tire in a fight compared to factors like how healthy they are (muscle:fat ratio and whether they are elderly), and how heavy their armor is.

For example a big hulking brute might have 1 attack with a big sword, but it takes a lot of swordplay or maneuvering to get that opportunity to swing.

A small lightly armored character wielding the same sword might tire fractionally quicker on the swing, but find it easier to get in close and take a swing.

The only thing that is slightly unrealistic is that a dagger and a huge axe are equally tiring, but the amount of fatigue the actual fatique is over the course of 1 round of swordplay can be rounded down to zero.