I can't speak to world builders in particular, but I do know of a particular fictional universe of world builders:
Myst. And I'm highly disappointed that this question has been here a year and nobody else brought it up yet.
I think the universe described in Myst (particularly the novels) points out important elements that would arise in a society of world builders. After all, we're not talking about a group of people who can only imagine worlds; they can actually go there. That changes things.
A lot.
If you dig through the Myst lore (ie: if you're willing to suffer through the sometime mediocre writing in the novels), you find out all kinds of interesting things about the D'ni, the people from whom Atrus is 1/4th descended. The people who invented the Art, and the people who built a culture around it.
They lived in a cavern 3 miles beneath the Earth's surface. Why? Well, if you have the ability to go to anywhere more or less at will... does it matter all that much if your home is a cave? You don't do your farming in a cave; you have bountiful harvest worlds for that. You don't strip-mine your cave; you have mining worlds for that. And so forth.
If you are always just a few steps away from any world, the one you live on starts being less important. There were public recreational ages, acting much like parks and so forth. Many D'ni nobles had their own private ages, analogous to owning a ranch and lots of land.
But more than this is the relationship between the D'ni and the peoples of the worlds they "create". Indeed, the quotes there are very important, because it is the ultimate D'ni heresy to claim that using the Art of Writing actually creates anything. They say that it simply links to an already-existing world.
That important, because everyone, everyone who starts thinking that they're creating worlds rather than discovering them immediately wants to lord their power over the native populations of that world. So my interpretation of the D'ni's views here is that it's a conscious effort to not start seeing other worlds as subjects waiting to be enslaved.
And I think that is probably where real world-building starts running afoul of human nature. Because if you really can build a world... what happens if you build life? How do you treat that life? Is it OK to create people who exist solely to serve you? Well, what if you create them such that it is their nature to serve you? Do these world builders have that level of control (notably, the D'ni do not)?
What if the life you create starts doing things you don't like? Is it OK to destroy what you've created?
If you're going to create a world where world building is a legitimate thing people can do, this is a question that absolutely must be answered. Do they treat them as casually as a novelist, who creates people that go through horrific torments in some cases, even casually discarding a character who has served their purpose? Or do they treat them with the respect befitting a world?
Or is it some of both? How much of both is it?