Perhaps more fatigue is not a good option. It was just a offhand alternate suggestion.
I think the hit penalty is better. Atm. the military pick just feels very strong early to mid game, even as initial weapons for lvl5-6ish mercs with spec in another weapon. Ofc. after hitting a Destroy Armor, you switch to Axe/Sword for high HP dmg or a 2H.
I think -5 hit penalty without Spec would make the ability a little more balanced.
Atm. theres almost no point spending fatigue on the standard hammer attack. Might as well save the fatigue for Destroy Armor on another target if your current one is busted already and let someone with another weapon do the finishing job.