TL;DR
There are several important factors considered when trying to determine if a group of stars are actually from the same stellar cluster, moving group, or association:
- Kinematics. Do the stars all appear to be moving in roughly the same direction, with similar velocities?
- Age. Are the stars all similar in age?
- Composition. Do the stars show relatively similar elemental abundances?
If stars satisfy these three criteria, it should be safe to assume that they all formed together, and were gravitationally bound at some point (even if they aren't today).
Types of kinematic groups
At the most basic level, you're asking about how to figure out if a set of stars are part of the same kinematic group or kinematic association, meaning that all of the stars move together, even if they aren't gravitationally bound over long timescales. There are several different types of kinematic groups - and here, by the way, I exclude globular clusters.
- Open cluster: A cluster of stars that formed together and are gravitationally bound. Open clusters are generally very young.
- Stellar association: A very loose cluster of stars, gravitationally unbound, that formed together. This is often a more evolved form of an open cluster.
- Moving group: Essentially the same as a stellar association, although usually older and perhaps more dispersed.
What you're talking about is a kinematic group - a term I'll use to describe both stellar associations and moving groups. What matters is that the stars formed together and are moving together - age doesn't really matter, although I assume you're talking about older stars. By the way, particular subtypes of kinematic associations are OB associations (composed of massive stars) and stellar streams (the remnants of a satellite galaxy destroyed by tidal forces).
Kinematics
These kinematic associations should display the same proper motion - that is, they should appear to stay together as they move across the sky. By determining their radial velocities - the components of their velocities towards or away from Earth - astronomers can compute the motion of stars in three dimensions.
This is commonly used to find the members of a stellar association or moving group, and can be simply done, using a telescope and spectrograph - although it make take time to notice significant proper motion. It can also be applied to smaller systems; for instance, Deacon et al. (2016) showed that 2MASS J2126-8140 is likely bound to the red dwarf TYC 9486-927-1. This involved studying the motion of the $\beta$ Pictoris moving group, the Tucana-Horologium association, and the TW Hydrae association.
Method:
- Observe stars for a number of nights to track their proper motion.
- Take radial velocity measurements using a spectrograph.
- Deproject the results and create a three-dimensional model of the stars' movements.
Part of Figure 5, Deacon et al. (2016). The motions of three kinematic associations considered (Tucana Horologium association in green, AB Doradus moving group in red, and Beta Pictoris moving group in blue), along with 2MASS J2126-8140 (purple triangles) and TYC 9486-927-1 (tan diamonds).
Let's say that the change in the star's right ascension per year and the change in the star's declination per year are $\mu_{\alpha}$ and $\mu_{\delta}$, respectively. Then the velocities in those two directions are
$$v_{\alpha}=4.74\left(\frac{d}{\text{parsec}}\right)\left(\frac{\mu_{\alpha}}{\text{arcsec yr}^{-1}}\right)\text{ km/s},\quad v_{\delta}=4.74\left(\frac{d}{\text{parsec}}\right)\left(\frac{\mu_{\delta}}{\text{arcsec yr}^{-1}}\right)\text{ km/s},\quad$$
where $d$ is the distance to the star. We can determine the radial component of the velocity, $v_r$, by
$$v_r=3\times10^5\frac{\Delta\lambda}{\lambda}\text{ km/s}$$
This can be calculated by measuring the change in wavelength ($\Delta\lambda$) of the center of a given spectral line ($\lambda$) due to the Doppler shift. Note that corrections because of a binary star must be applied.
Once you know $v_{\alpha}$, $v_{\delta}$ and $v_r$ for a given collection of stars, you can check and see if all the stars have similar components. If the distance is unknown, you could even just look at $\mu_{\alpha}$ and $\mu_{\delta}$ instead of $v_{\alpha}$ and $v_{\delta}$.
Age
It is often assumed that the members of a kinematic association formed at roughly the same time, although multiple discrete waves of star formation occurred. The age of an individual star can be determined in a few ways. These normally involve measuring its mass and other properties, and modeling the evolution of such a star over time, comparing the model at different ages to the target star.
Other methods, like gyrochronology, do exist, but I'm not sure how widely they're used.
Another approach involves studying the age of the putative moving group as a whole. This can be done by plotting the stars on a Hertzsprung-Russell diagram. If the group is old enough, some of the stars should have left the main sequence, forming what is called a "knee" on the diagram, or a "turnoff point" - the point on the main sequence beyond which all stars in the association have evolved off the main sequence. If a star appears beyond the knee, then, barring the possibility of a blue straggler, it is unlikely to be a part of the group.
Method:
- Determine the mass of the star if possible, often by observing its motion and that of a binary companion.
- Take spectra of the star and determine its other physical properties.
- Model the star's evolution over time, and find its most probably age.
- Compare the results from all of the stars in the possible association.
A Hertzsprung-Russell diagram for the globular cluster M3. Note the knee off the main sequence and the bunch of blue stragglers at the left. Image courtesy of Wikipedia user RJHall under the Creative Commons ShareAlike 1.0 License.
I can't give you any easy formulae to try this one out on your own; this method involves computing stellar evolutionary tracks, and those are fairly complex. However, a number of codes exist, and some are available for download. MESA is a common one used by astronomers, and can be downloaded and used by you, if you want. I haven't tried it out yet, but I know it's commonly recommended.
The pre-generated Geneva grids are also widely used (see Eggenberger et al. (2008)). You should be able to access them through this page, which gives a brief description of each. The Geneva models are also commonly used; most importantly, they cover a broad selection of stellar masses and metallicities, so given a certain star, you might be able to find a model to match.
Composition
The stars in a kinematic association likely formed at the same time and from the same molecular cloud or cloud complex; therefore, you would expect their chemical compositions - and, in particular, their metallicities - to be the same. With the exception of chemically peculiar stars (arising from various processes), the group of stars might show the same spectral lines, for the most part - with expected variation in strength depending on temperature (and thus spectral type). The metallicity in particular is the key property, and can be readily measured via spectroscopy (spectroscopy, by the way, is the underlying theme for today).
Method:
- Use a spectrograph to determine the spectra of the set of stars, if you haven't already.
- Sort them into subclasses of their spectral type (note that for, say, OB associations, most of the stars will be of very similar spectral type).
- Look at the presence of spectral lines to determine elemental abundances, as well as metallicity.
I'm less familiar with the criteria for deciding for sure whether two stars are part of the same kinematic group, and I can't point you to a whole lot of stellar spectra databases, but I know that the ESO has a solid library of spectra for a variety of spectral types, as does the Sloan Digital Sky Survey (albeit for individual stars). Looking at spectral databases should be a starting point for you; reading spectra can be a helpful skill.
Final notes
One thing to keep in mind is that there's going to be variation here. Chemically peculiar stars, multiple-star systems, additional waves of star formation, and other problems will exist, and can contaminate your data set. That said, you can presumably remove these edge cases or adjust for them in your analysis. Plus, with the application of three different methods, you should be able to find that each member satisfies several of the criteria, if not all of them.
If I can talk about the example you gave (the Sun vs. Sirius vs. Alpha Centauri), note that Sirius has an age of only a few hundred million years. The Sun, on the other hand, is roughly 4.5 billion years old, and Alpha Centauri is about 4.5-6.5 billion years old). Sirius and the Sun definitely did not form as part of the same stellar association, and Alpha Centauri also likely formed separately.
We can determine a few things about the open cluster the Sun was born in, but it's harder finding the cluster's former members, largely because of the passage of time. The open clusters into which stars are born dissipate on timescales of maybe $\sim10^8$ years, meaning that there should be very few traces of whatever cluster formed the Sun. Therefore, all of the techniques above work primarily on young groups of stars - with ages in the millions of years.
If you're looking to find the radial velocities and proper motions of stars in our galaxy, you're in luck. The second data release from Gaia came out this spring. It contains the positions and proper motions of 1.3 billion stars and the radial velocities of 7.2 million, meaning you might have enough data to determine if a number of stars are moving together.