I would expect any early settlement/colonisation to be haphazard, with everyone grabbing whatever they think is valuable and fighting (with violence, or in whatever political or judicial arena is available) over the legitimacy of their control.
Over time, control would be consolidated. Far-flung patchworks of installations would separate into distinct entities, lose parts to others’ control, take control of intervening ground, or all of the above. Stronger powers (perhaps political entities from Earth) would establish their authority over the weaker ones, if they didn’t have it already. Between the stronger powers, some form of consensus would emerge, tamping down most (but not all) disputes.
(The alternative is for a centralised or consensus authority to be behind the colonisation efforts from the beginning, but that’s boring!)
I base this on three kinds of historical precedent: colonial boundaries, farming/mining claims, and modern laws.
Colonial boundaries
Late in the colonial era, the major colonial powers had reasonably well-established rules for who held what and how to partition the land. It looks orderly, well regulated, almost civilised… until you remember that they were still conquering inhabited land.
Example: The Scramble for Africa. Literally the entire continent ended up assigned to one European nation or another (with the possible exception of Ethiopia, which remained independent except for a short Italian occupation in the 20th century.)
Earlier, though, the main determiner of colonial boundaries was what you could effectively hold. As a result, borders were… fuzzy. Sometimes a colonising power would claim a large area, but only effectively control small portions, so other powers would come in and settle their own colonies in the claimed area.
Example: The Treaty of Tordesillas established a Spanish claim to the entire New World (except a bit of South America that they may not have realised crossed the line). Great Britain, France, the Netherlands, et al. didn’t exactly care.
Other times the original power would extend their control through the territory they already claimed, typically splitting it into new colonies in the process.
Example: New South Wales, the British claim in Australia, started as half the continent and gradually had six new colonies split off from it (though one, North Australia, was promptly merged back in).
Sometimes colonising powers would claim land their explorers hadn’t even surveyed yet! Colonies might have a defined extent along the coast, and then stretch an indefinite distance inland along an effectively arbitrary line of latitude or longitude.
Example: Early British colonies in North America sometimes had charters “from sea to sea”, far beyond their actual control. This was wishful thinking, of course, but it seems like the first real modification came from French claims on the centre of the continent (which the French explored). The British colonies still extended much further west than any of them had actually settled until 1763. (After that it reversed, and colonists started settling further west than colonial authorities had claimed…)
And of course colonies would conflict with each other, dispute boundaries, claim each other’s lands, and just plain change hands. Even colonies of the same colonial power!
Example: See the links in the previous example for arguments between British North American colonies over who held what. Among the many colonies that changed hands are New France, New Netherland, Spanish Jamaica, Portuguese Bombaim, German New Guinea…
Farming and mining claims
Just because a sovereign entity has claimed ground and fended off all challengers, doesn’t mean its use or ownership is settled. Someone has to actually go there and do something with it. Historically, there have been several occasions where individuals or groups could claim a chunk of land, typically for farming or mining purposes. This leads to all sorts of conflicts and shenanigans.
Examples:
- In the United States, several land runs (or rushes) opened up land to literally the first person to run (or ride a horse, or drive a cart) to it.
- In Australia, similar (if less dramatic) allocations of land were made to selectors, which sometimes meant conflict with the unregulated squatters who had already been grazing livestock there.
- During the US and Australian gold rushes, miners could claim land under rules that were generally accepted and enforced by the communities themselves. The governments later got in on the act, and to this day the ability to stake a “mining claim” exists in some form in both countries.
- Of course, anywhere that you have claims, you have claim jumping—the attempt to take over another’s claim by force, subterfuge, or simply by moving in when the claimant fails to make use of the land.
Modern laws
These days there are a lot of rules over who owns what, and what “ownership” really means.
It was a long-established principle that whoever owns a piece of land also owns the underground below it and the air above it. Both of these are changing in modern times, though.
“The air above it” only became an issue in the age of air travel, and governments had to start coming up with air rights to say who can do what in the space above the ground.
In theory, I think it’s still the case that “the underground below” belongs to whoever owns the land. The problem there is that the real “owner” is the government (in England and some other Commonwealth realms, the Crown), and the householder living on the land just has a right to use it. I’m not a lawyer, but I believe the term for “really owning the land” is allodial title, whereas just having the right to use the land is “freehold”.
Off the coasts, the modern Law of the Sea establishes several zones of control with varying rules. (Who has the right to sail through? Who has the right to fish? Who can mine the seabed?)
I joked at the top about it being “boring” if state-like powers are in control from the beginning, but actually there’s still a lot of scope for conflict within this kind of rules-based regime.