I can confirm this happens.
I also lured bandits or whatever to that swamp/port village and helped the militia to take them out, but there were several casualties for me and them, I can’t remember exactly what happened. Now, whenever I come close to that village, I can see that there are always some spawned troops of militias there. (“Oh no, here comes those troublesome mercenaries! What beasts are they bringing this time?”)
It is a village that belongs to a faction, by the way, but somehow it also has its neutral relations thing when you hover the mouse over it. Maybe another bug there.
Save file attached. Village name is Meierwik just west of current party location.
.