I think it sounds very strange to say, “…and the gods know what else…”.
(The saying is ‘only God knows what else’)
If you want to play off that saying, I’d suggest changing the red underlined portion to “… brigands and only the gods know what else are terrorizing these parts…”
The reason this sounds odd is not only because of the alteration of that saying, but also because you re-use “know” in the same sentence (repeated descriptive word usage in a sentence is typically bad, i.e. I wouldn’t say, “We know that the Necrosavant knows that we’re right here”).
Perhaps try to replace “We both know…” with something like, “It’s common knowledge that…”.
Switching “…and they all are quite fond of the roads.” to -> “…and they are all quite fond of the roads.” would sound much better, as it is better syntax.
You don’t need to use the word “the” for “…guards lost the heart…” (Though I suppose you could use “the”).
I think it’d be better to use “…my old caravan guards have lost heart for the job.”.
Lastly, these are only suggestions…
If you ever want to reach out to me for any assistance on grammar/syntax, etc, I’d love to help! I’ve sent Overhype Studios a friend request on Steam (My username on there is “That One Guy”).
"It is not death that a man should fear, but he should fear never beginning to live." ~Marcus Aurelius
Game: "Characters with a height advantage against their opponents are harder to hit"
Me: "That's not true, and my short axeman is living proof!"