You appear to be asking two questions:
How can I justify slavery on this planet when it is illegal in the rest of the galaxy?
Can the slaves do anything about their condition?
Occam's Razor: All things being equal, it's impossible to maintain slavery
In the beginning was tyranny. It's relatively easy to maintain slavery under a tyranny. Those who dislike what the tyrant says are eliminated. Those who provide for the tyrant's needs (no matter how they do it) are supported.
As time goes on philosophers, statesmen, rebels, sympathizers, do-gooders, etc. come and go. A few wars are fought, and over time laws are passed that slowly begin to give The People greater political power. It gets harder to treat people "unfairly" (and that's in quotes because "unfair" is perhaps the most subjective word in the known universe, see the cartoon in Renan's answer for a good example).
Eventually you end up with a representative government where even an individual (with enough organization and effort) can be heard loud and clear. An no matter how valuable or necessary slavery may seem, that one person can start a wave of sympathy that, combined with a representative government, eventually leads to the eradication of slavery.
In other words, all things being equal, it's impossible for slavery to be sustained. In fact, as government becomes more representative, the shorter the period of sustainable slavery becomes.1
So, all things aren't equal. Something's upsetting the balance.
Frankly, a valuable or critical resource is not enough to justify slavery. During the U.S. civil war cotton was a valuable and critical resource. So much so that the South refused to ship whole docks full of it to pressure England and Europe to enter the fight on the side of the South. It almost worked.
However, it's difficult to imagine a resource so critical that it could work. To make things worse, the more critical the resource, the more it works in the slaves' favor. Go on strike, everybody dies, but the whole honking universe knows what happened because everyone was inconvenienced, enter the sympathizers, laws mandating change suddenly occur because, above all else, it can't happen again. We need something much more unequal (call it unfair) than this.
Worse, yours is a galactic empire with technology out the wazoo. You need a reason to justify no robots. You don't need full AI to dig minerals or cut down trees. In fact, a case could be made that half the jobs out there require no imagination at all. They're completely procedural and don't have automation today simply because the flexibility of legs, arms, and fingers have yet to be mastered by the tech. So we need something unfair and unautomatable.
Finally, slaves only make economic sense when it's cheaper to use people than anything else. Building pyramids with slaves today would never be economical because one very well paid crane operator is worth thousands of untrained people and the cost to maintain them (costs to maintain slaves can be mitigated by inexpensive replacement, but only to a degree). On the other hand, we still use low-pay workers to pick fruit, berries, and other food stuffs that machinery simply can't do yet. Long story short, we need something unfair, unautomatable, and economically viable.
Any unobtainium will do, but it needs to be unique to this planet. On a galactic empire scale, that would suggest a very large world so that a great deal of material can be collected at once. This makes it economically viable but hard on people (heavy gravity).
It can't be mining. Mines run out (it's difficult to imagine, but you can deplete a planet). That means a renewable something. A biological substance that is unique to the planet. The chemistry of the ground and atmosphere could be duplicated... so what would be hard to duplicate?
A nearby pulsar, with a whomping strong magnetic pulse that regularly sweeps the planet.
And that strong magnetic field in combination with the unusual chemistry of the planet produces... oh, say a tree sap that happens to be the only life-extending/fully-rejuvantive analgesic known to humanity.2 Better still:
No automation. The rotational period of the pulsar is frequent enough and the magnetic pulse strong enough that you'd be constantly repairing or replacing anything that had electricity-bearing wires. The cost of automated operation would be enormous if it was possible at all. I'm assuming the average magnetic pulse from a pulsar would be impossible (if only economically) to shield.
Difficult synthesis. Do-gooders can't simply say, "let's set up a factory that duplicates those conditions over on Tatooine!" because even if you could create the delicate combination of chemistry and magnetism that creates our biological unobtainium, the cost would be worse than just repairing and replacing the automation. Remember, pulsar, and little Johnny's "cosmic science kit" really can't duplicate that level of strength. So... no can do.
That leaves people. And a lot of people, because you have an entire galaxy to supply and you're the one and only location that can do it. The environment may require environmental suits, but you can't have anything electrically driven on them or the magnetics will destroy them. You can't have a lot of metal at all or you'll kill the human via inductive coupling. That means an environment that needs no more than hazmat or scuba suits to harvest the unobtainium.
And since profitability is always an issue, you're either paying them next to nothing (keeping them in indentured servitude like the company towns of the U.S. early 1900s), or it's flat out slavery where pain avoidance is the primary motivator for work.
Can the slaves do anything about this?
As I mentioned in the beginning, no society with any form of representative government will tolerate slavery for long because there will always be somebody who sympathizes with the plight of the enslaved.3
Therefore, there will be a constant propaganda war going on to keep the galactic populace distracted about the real conditions on our slave planet. This will be a large, complicated, and sometimes illicit effort that will be a great sub-plot in the story.
There will always be someone on the planet itself, one of the not-enslaved, who sympathizes with the slaves. These are dangerous people, because they will be planning (a) whistle blowing events, (b) armed rebellion, (c) aid societies and all kinds of solutions to try to help the enslaved. Don't underestimate these folks. Grassroots anti-slavery movements were a huge contributor to the U.S. civil war.
Finally, there's the slaves. Slavery works because the enslaved are (a) emotionally drained (institutionalized), (b) physically drained (you only want them strong enough to work, not strong enough to fight, (c) disenfranchised (no one anywhere near them likes them, no "nearby" support), (d) unempowered (everything is provided for them, an entire welfare state, the threat of leaving that condition is frightening), (e) and controlled (regular oversight by armed and authorized agents).
Overcoming slavery in your condition would likely begin with an underground railroad that smuggled out both slaves and evidence of the slavery to encourage sympathy amongst the politically powerful outside the world. But whether or not they're successful will depend on whether or not your society is truly free — free to develop abolition societies or allowed to have religious beliefs that preach the equality of all souls before diety. It's a complex mix — but that's what makes a good story.
1 So long as small groups of people can hide in the political shadows, and so long as there's an illicit desire that people are willing to pay bookoo bucks to fulfil, there will always be some slavery. Sex trafficing and the drug trade in our world today are a good examples. However, I'm assuming that organizations this small are outside the scope of your question.
2 Nothing drives humans to spend money like the fear of death. With the possible exception of professional sports.
3 You could always declare your galacitc empire to be a monarchy that doesn't tolerate free speech, but that's a very hard thing to do without a lot of sycophants. It took full engagement in a popular war-of-revenge for Hitler to convince the people to give him full power. After a while, that power could only be held by fear. And even that was beginning to erode by the end of the war. Generally speaking, people will always want a voice, and the cost of suppression probably goes up exponentially with the number of people you need to suppress.