Armor doesn’t protect you from getting stunned, or feared or other status effects. As sky said, relying purely on armor (esp for most of your men) is not the best of ideas.
One issue I’ve found with the Brawny trait, however, is that it doesn’t seem to reduce the penalty from a helmet. Which is a huge issue, since I try to have helmets as 80% as durable a mercanary’s body armor.
