Not a feared army
Harsh environmental conditions do not lead to a feared army. Fighting does. At best, harsh environmental conditions make the army feared in those conditions. And the truth is that historically, invaders haven't been that afraid.
Consider the case of Russia. It has harsh weather which makes it difficult to support supply lines during winter. Yet I can easily think of three invaders who've made it to Moscow and sacked the city. Two of them (Napoleon and Hitler) ultimately failed. Why? When snows cut off their army from their supply lines, their army fell apart and retreated. This despite the fact that they regularly won their military engagements.
The successful invasion of Russia was by the Mongols. Why? Because they were perfectly happy breaking their army down into units small enough to live off the land during winter. That was much closer to how they normally lived. But that only worked because neither the Mongols nor the Russians were particularly advanced. If the Russians had had more modern defenses, the Mongols wouldn't have been able to defeat them. If the Mongols had more modern weapons with a need for resupply, they wouldn't have been able to fight after losing their supply lines.
You can see the same thing in your Dune example. The natives weren't a feared army. They were ultimately successful, but it was the innovations that Paul brought that led them there. In particular, his threat to destroy the spice. Up until then, the invaders fully expected to be successful in subjugating the natives.
The Dorsai model would be possible, but you seem to be excluding that. There the world hired out its male population as soldiers. This made them feared militarily.
Why do people live there?
You identify the area as not growing enough food and dependent on trade. Why does anyone live there then? Trade is a luxury for other countries. So when other countries experience a shortage of food, they will stop trading. Everyone in this country dies. In medieval times, this would be common. No one would actually live in this country. After the first food shortage, everyone would die or leave and there'd be no reason to return to a landlocked country where travel is difficult.
It would be different if this country were somehow essential to trade because travel through it was easier than alternatives. Then the country would repopulate when trade reopened. But you excluded that.