You need more men. Even cheap 80 gold ones armed with just a spear. They are meat shields and allow you to hit harder targets for a better pay off.
And yeah, the humans don’t build new settlements. But impoverished settlements don’t have any bearing on the available contracts which are the primary source of income for a long while. And the map is decently large if you go out into the wild lands, far from human settlements. But I wouldn’t advise that until you have a full roster of level 8 or above.
If you just want some money to build your roster, caravan contracts are perfect.
And lastly, the “discovery” quest locations will never be “Raze” quests, just not programmed that way.