Well I can’t really speak how they are late game because they usually stop me from getting that far. I usually encounter just a one or two of them 40-60 days in and that’s it. They just slaughter everyone, and yes I know to put your best guys on the corner and what not. I mean the people I’ve had from the very beginning or close to it couldn’t 2v1 them, never mind trying to 1v1 them. Just having them shield wall to buy time while all the other enemies are being killed doesn’t work either. The fact that their melee skill is on crack kind of prevents the shield wall from working reliably.
You probably need better armor. They should deal big armor damage but not 3-hit someone. Once you get scale armor and maybe battle forged perk it should look a whole lot different. If you feel you are not up to take them, just retreat. There is no point in fighting a battle that you can’t win.
I think the main problem lies in how the scaling works right now. If you roll a good map and get decent guys and armor from the beginning, encountering 2 savants at day60 will not be a problem. But if you lost some guys, your map sucks or you take 3-day caravan escorts you may at the same day ingame just be wearing a few mail hauberks maybe (e.g have 30k crowns less than in the other campaign). The game felt a lot more fair and transparent when it was still balanced on renown imo.