World-building is (in theory) an infinite task. By typing that question you were building this world, and by writing this answer I'm doing that too. I think it's important to first realize that we are always building as we go, no matter how much we want to do before we start "writing stories". That being said, I like to think of the process of world-building being broken up into two steps; world-building and world-populating.
World Building
Define some yes/no world rules across a range of areas (see below) that are accepted truths to you, the builder (and your team of builders). These are things you know to be true in the world. You need to work with constants at all times when building. This is not the time for hypothetical answers that your characters might offer to the same questions, because you shouldn't have any characters yet. The key point to this foundational step is that, it doesn't matter what the answers are; what matters is that you know what the answers are.
Leave Philosophy At The Door.
Areas you might want to consider:
Cosmology:
Star type (main sequence or a later stage star?)
Planetary position (in the circumstellar habitable zone?)
Planetary movement (moving towards the star or away from it?)
Planetary:
Planet type (Earth analog, gas giant, iron core?)
Planet diameter
Atmosphere (Is there one?)
Spiritual:
Deity Existence (Do divine beings exist?)
Deity Scope (Universal?, Planetary?, City-sized?, Smaller?, Or do they having a different sorting order altogether?)
Evolutionary system (Adam & Eve, Darwin?, Lamarckian?, Something Else?)
It's difficult to pin down an exact set of areas to divide possible questions up into, because the way you categorize things might be slightly different from person to person.
For example: "Is climate part of physical geography or separate?" "Well that's semantics, isn't it?"
Either way, you'll most likely look at both climate and physical geography during your world-building.
World Populating
Populating your world is giving life to it. I like to think of it in a very binary style of objects and events. Objects are things that appear in the world, events are things that give my objects a reference to time. My characters, species, regions, settlements, food stuffs, political states, deities, rivers etc. are all objects. Their births, deaths, battles, coronations, adventures, floods, foundations, destructions are all events.
IMHO, world populating comes after world building. Populating before building could lead to inconsistencies down the line. You might want that, but if you don't, it's a real headache.
Timeline:
Consider this one early on. Time is constant, the numbers and dates we mention are just overlays or masks to help us understand it more easily. A timeline will help you get a sense of perspective for where your story/ies will be taking place in the history of the world. Start with the largest chunks and then subdivide to give greater detail. In the real world our largest chunks are special (species) related. Dinosaurs(saurians), followed by mammals. You might want to do something similar. We then tend to divide this on sub-levels based on technological/evolutionary/physical developments; stone, copper, bronze, iron, steel, renaissance, enlightenment, industrial, digital etc. It makes no difference how you do this, because at the end of the day you're only ever counting seconds :).
General To Detailed:
Populate your world like you would paint a picture. Sketch it out first, then add greater and greater levels of detail in a logically sound order. If you want to add a character into your world and say they are born in region A at this time, it would make sense to already know what the majority species living in that region at that time is. Is your character therefore fitting the mould or from a minority?
Start at the beginning of your timeline, and sketch out where your species are going to be. Work through your timeline and move them about a bit. With the movement of your species, you're ready to look at cultures & languages. Different cultures and languages (in theory) come about due to the break-up and dispersal of groups of a species to different locations, who then lose contact with each other long enough to affect their customs & speech. If you follow this theory, you can't comfortably know where and when you will have different cultures & languages until you know how your species spread, separate and come to settle.
Warning: you may not want to worry about this level of thought & detail. But for those "purists" out there, the spread of your species across your world will determine how you design where your cultures & languages originate from and they themselves spread to.
There are others who will say jump right in and have a go, and that might work for you. I think it all comes down to this question: Is your world for one story or for many? If you're only going to tell one story from one point in time, your world is 2D, and you can jump right in and start from wherever you like. If you are planning on having multiple stories, hundreds or even thousands of years apart, you've got a 3D (or perhaps MD: multi-dimensional) world. The stories that happen chronologically later need to agree with those that happen chronologically earlier, but there is no guarantee the order you write them in is the order they happen in your world. If you're planning a 3D/MD world, I recommend following the example above.