The first step in this process is to define the alien's body plan, life cycle, and the nature of its planet of origin. This is an absolute prerequisite to determining its psychology, though it may be possible to reverse-engineer the former from the latter.
For example, one alien species I created came from a world in a trinary solar system with relatively eccentric and precessing orbits, where the seasons were so variable that on an evolutionary level they were completely unpredictable; only with the rise of sentience could then be predicted, and even then not easily.
The effect this had on these creatures was that they had evolved a psycho-physiologic need for unpredictability in their environment, that prolonged periods of predictability would lead to levels of stress that would lead to derangement and violent outbursts. In consequence, they had a unique profession - the stealth decorator - who would sneak into a home (that they had been contracted to enter) on a random schedule and perform some redecoration that could range from trivial to major, thus providing a necessary element of unpredictability in an environment that civilization had made all too predictable. Given their propensity for travel and exploration, these creatures were nicknamed Tourists.
Another example of mine was the Hive Trees. These are tree-like creatures in a science-fantasy universe. While usually sessile (they are able to move on occasion), they have the ability to grow numbers of separate mobile forms, in an almost infinite number of variations, that have an instantaneous link back to their parent tree, these remotes are effectively part of the parent tree even though they are physically separate. The tree houses the brain, so the loss of a remote is not particularly great, sort of like losing some hair or fingernail, or a little more serious.
The psychology of these beings was a weird combination of paranoia and bravado - they would risk their remotes like pawns in a chess game, but the more control their remotes have over the tree's surroundings, the more paranoid (by human standards) the tree becomes of its personal safety, not allowing anything other than its own remotes to come near, while trees in hostile areas are more willing to allow dangers to approach more closely - This difference is mainly due to effective area of control.
The variations are infinite, but as long as a world creator defines the species' world, physiology and capabilities sufficiently well, extrapolating a psychology is mostly a matter of logic.
Unless an alien is radically different from human, it should share at least some common psychological background, such as the need for sustenance (even if it is an autotroph), the need for self-preservation, the need to reproduce, et cetera. However, there can always be exceptions. Imagine a species that has a life stage that cannot eat (like a mayfly), that is primarily reproductive, and must mate quickly and produce eggs before its inevitable and imminent death. What about a species that has a caste of individual that cannot reproduce, and is responsible for defending the hive. While these are real-world examples, what if a creature with similar traits was sentient? How would it think and react?
