You can't be building a world like this, without intending to draw parallels against our own world. And our own world is not fair.
If you are aiming for fairness, you are worldbuilding a world of tedium and boringness. In the real world, whatever system you find to allocate fairness will be found unfair by some.
- People should be paid proportionally to how scarce their skills are!
- People should be paid the same per year!
- People should be paid the same per hour!
- People should be paid the same per gram of bodymass!
- People should be paid the same per dollar earned for the company!
- People should be paid the same per year they've worked for the employer!
- People should be paid the same per... what's fair?
Fair, as others have said, is obviously an illusion, and in reality people usually try to be fair, so will find some mix of the above, and more, to try to ensure their staff are looked after.
And in reality there will always be people who get the short end of whatever measuring-stick is applied, and will say it's not fair.
Building a world where this issue does NOT arise, is building a bland, beige world. Instead, I'd argue to build a world where these issues are accepted as part of the world's fabric: hardships to be borne, occasionally argued about or overthrown when it chafes too hard, like the caste and class systems that we see on Earth today.
Strikes and industrial action are a good way to explain the non-availability of resources that might otherwise give your characters an easy solution. Oh, no, the car broke down. Why not just call a taxi? Well, there's public transport strike demanding more sections of secure tunneling for the smaller drivers, after that accident last week...
But it doesn't have to be plot-relevant, it's also just a good way to give the story color, whether it's as a casual reference to how the tiny guy gets to boss around a whole business empire of giants, or how the wealthy giant gets to be constantly groomed by a cloud of little people, or whatever.
I wouldn't make wealth be absolutely linked to size, any more than it is to race or gender today. But I'd pick a size and make them the ones with the privilege, whether it's because they can do more skilled work, or are bigger and stronger, or whatever.
You can also play with people's internal stereotypes. You can set up the giants to have everyone assume they are ignorant, but then find an educated one who explains that sure, they move slow, and talk slow, but not because they think slow. Just because momentum, and caution not to hurt those smaller than them. As for the rest, it's caste-related; nobody writes textbooks large enough for the giants, the ivory towers of educational institutions are built to a scale that excludes them, etc etc. Or vice versa: the little ones are ignorant, not because they have tiny brains and lives too short to learn anything, but because they can't lift the textbooks. Entrenchedly bigoted stereotypes between the sizes allows both for lively humor, but also for deep commentary.
Embrace harsh social realities, but only enough to chafe, not to cut.