It seems like civilizations that used the atlatl usually switched to a different melee weapon, so the atlatl and its darts were more exclusively ranged weapons.
If I am designing a civilization that uses atlatls, could they possibly use the atlatl (thrower) as a smaller weapon in itself? Would this make it less effective?
The method I have in mind is literally a hatchet blade fixed to the end of an atlatl.
you want the end of the atlatl where the spear sits to be light to transfer the most energy to the spear when throwing.
If you put a heavy blade on the end then after throwing you end up needing to deal with the energy that you imparted to the atlatlchet. Energy that didn't go to the spear's speed.
This is going to run into the same mechanical problem as putting a spear blade on the end of a bow. Like a bow's tips, the tip of an atlatl needs to accelerate as quickly as possible to get the velocity of the projectile as high as possible. Sticking heavy things on them impedes this.
Now, that said, with an atlatl you have the advantage that there is only one tip, so the handle part could maybe have an axe head or something attached to it. It will need to be shaped so as to not endanger the user when throwing things with the weapon, and the thin, light shaft of the atlatl won't be nearly as good as an actual axe handle, so you'll have to use a short grip on it and then the long end of the throwing half will get in your way a lot... But it would technically be better than fighting bare-handed.
However, a dedicated hand axe will work a heck of a lot better and not risk damaging your expensive and finely-tuned atlatl by whacking armored opponents with it. The extra weight is pretty negligible. You'll want a hatchet for doing camp chores anyway.
Alternatively, I suppose you could put a hook on the back of a hand-axe suitable for loading darts on. It wouldn't throw anywhere near as well as a real atlatl, but up it to javelins instead of darts so they rely more on weight than speed and it would probably buy you a few extra yards of range.
Despite searching, I can't find anyone who has done this. The only weights you can put on an atlatl are about mid-shaft and then only about 85 grams. The shaft of an atlatl, to be efficient with such lengths, seems to need to be as light as possible.
So to make it a melee weapon, we'll need to be creative.
In any case, your shaft needs to be made of something REALLY strong and REALLY light. Even with that, it's likely that the kind of abuse it will take as a melee weapon will damage the spear-throwing ability. A shaft of normal materials might still work as a club and an atlatl, but it would likely do one job, the other or both poorly.
Have you considered a punch-dagger? The handle where the person holds the atlatl would have a grip that included a strong hold and a built-in piercing weapon. This would give the fighter an emergency weapon in desperate situations and look really bad-ass.
If the shaft were REALLY (handwavium) strong and flexible, the punch-dagger could double as the business-end of a spiked war club. Or an axe, if the punch-dagger were shaped like an axe head. In this case, there would be a grip at both ends. It would still only be a light melee weapon. A dedicated weapon would work better.
A thin spike on the end of a strong shaft could be used foil-like as a stabbing weapon. It would likely wreck the atlatl function, and the atlatl part might cause it to stick in an opponent. Or a spike on the handle end could work like a knife to stab (rather awkwardly).
A chain attached to the grip-end of a strong atlatl could be attached to a flail head. While it strikes me as awkward, most of the weight of the flail would be away from the arm. Then the mid-shaft weight could functionally be a second handle to allow the flail to be swung about two-handed.
I understand you would like to give your world some original nuance, but I cite the old say: "jack of all trades, master of none".
This is true also for weapons. The most lethal and effective weapons are those that are
optimized for one kind of usage (to put it simply)
easy to learn to use effectively
#1 allows to optimize the weapon in itself, #2 allows optimizing the training of a large army with less economical effort, hence optimizing the overall effectiveness of a group.
There is a reason why in all history most armies where composed by troops specialized with a single kind of weapon. In the cases where you really needed more versatile troops (such in more modern warfare) you had a set of different weapons each optimized for a different purpose.
For example, the ancient rome soldier had a gladius for melee and the pilum for one-shot ranged attacks.
Another example: in modern warfare the typical soldier have an assault rifle for normal mid-range operations, a pistol as sidearm and an assault knife, and probably some grenades. In a platoon you could have more specialized troops, for example those able to use an RPG or a stinger missile.
In general Swiss-army knife sort-of weapons are more difficult to use and to train to use effectively. Moreover, once the weapon breaks, you loose all your offensive capability, whereas if your sword breaks you could always use a dagger or a bow to be still useful on the field.
In addition, a more mechanically complicated weapon could have more mode of failures and could be more costly to build an maintain.
Bottom line, aside from ceremonial reasons (e.g. the halberd, a weapon that was notoriously difficult to use effectively, but was great for show), funny multi-purpose weapons are a big no-no from a strict practical warfare POV.
Plenty of excellent answers here, but I'd like to provide a note of practical experimentation that you might be able to wrap your head around.
Throw a baseball. Now throw a baseball with a 2 kg wrist-weight attached to your arm. You'll be lucky to get half of your top speed. I tried this with .5 kg wrist weights, trying to play Beat Saber, and it really got across how much a little extra mass slows down your actions.
A typical stone axe head weighs .3kg. The effort to accelerate that weight (regardless of which end of the stick it's on) will subtract from the energy that goes into the spear. You could make the axe head lighter, but then it would be a less effective axe.
You may want to take a look at Woomera for ideas. This traditional Aboriginal Australians' spear thrower often made as multi-purpose tools: often shaped like long narrow bowls to carry food and water, possibly can be used as shields, and many had a sharp stone cutting edge attached to the end of the handle.
Another to take a look is wood-with-embedded-edge weapons, such as Macuahuitl.
