That's not odd, nor unheard of in reality.
In human cultures who have/had polytheist pantheons, it was rather uncommon for people to worship all their deities equally, or even at all.
Whether you are taking Ancient Greeks, Nordics, Hinduists or Ancient Romans, to name a few, people worshipped mainly one or at most a few gods, mostly depending on their profession/cast and/or place of living.
Pantheons serve the purpose of offering specialized gods for specific elements of the world and abstract concepts.
For example, Greek soldiers had a major insentive in worshiping Ares, who was closely related with war and battle, or Athena, who was associated with war and wisdom. That's directly why Sparta, a very military city, was heavily worshiping Ares while Athenes, a city who valued having both a good army and excellent intellectuals, was entirely directed at Athena.
The Norse gods were also worshiped that way, and who interacted in which way with which god(dess) wasn't that much significant. Soldiers or farmers weren't supposed to have the same main deity as their Jarl (which would have looked quite silly and hypocritical, if I may...). That's a notion that appeared as we know it with Judaism and which was spreaded by the Christians.
In fact, Jesus was considered as another god when introduced among the Nordics, they mostly didn't care. And since Christians made it pretty clear that worshiping only the One True God was mandatory, and that converting other was the main goal, that's one of the reasons Christianism ended up winning the popularity game there : the local religion wasn't playing.
To sum it up, you can perfectly have people worship mostly one or a few gods from a pantheon of whatever amount you want to have in total. Just make them to be associated with distinct domains. A domain is a collection of abstract and concrete elements and symbols to recognize them easily, whether they are represented explicitly or directly by their symbols.
The attributes that should be present in your pantheon should be those that concern your civilization's culture. For example, they are not likely to have a deity responsible of the sea(s) if they live in the center of a wide continent, and if they do it will be a minor attribute or the domain of a minor god(dess).
I believe I saw other questions about that pantheon you are created but don't remeber much about the humans behind it.
Anyway, here are a few examples of attributes that could be in your pantheon's domains:
(some may not apply while some are fairly common to all cultures)
Concrete stuff : Earth, land, crops, beasts, fire, thunder, wind, rain, mountains, sand, water, metal, alcohol, wood, plants, fishes, weapons, poison, specific animals (horses, boars, whichever may have a particular value for social, spiritual or economic reasons), tools, ...
These are things that can be directly seen and eventually touched. They are usually seen as the direct manifestation of a deity, a gift from it to mankind, his/her own body...
Abstract stuff : War, beauty, wits, strength, sickness, life, death, fe(a)st, intelligence, politics, madness, patience, curiosity, fertility, metallurgy, arts, hunt, medicine, knowledge, luck, sex, fate, happiness, hope, craftiness, fishing, magic, ...
These are attributes that are administrated directly by a deity, or on which a deity has at least some control : Some deities were said to decide who would die, when and how, while others merely gathered them when their fate came. Those are the backbone of gods and what determines who will worship them and why : One will worship especially the deity handling war in order to be victorious in one while his/her spouse would be likely to call for the deity of luck/fate/mercy so that his/her beloved comes back unharmed. A city that relies heavily on fishing to feed would have one of its biggest temple dedicated to the deity of fishing and sea (if not the biggest at all) while blacksmiths in that city would be unlikely to care much about it.
After you've gathered your pantheon's domains, you can begin to split them among your deities. Keep in mind that they don't have to be limited to one and only thematic. A god of war, conflict, forge and metal makes sense, it adds depth. A god of war and politics too. They can even coexist in the same pantheon. A god of mercy and death also has a good depth : in displays the fact that the culture who worships it doesn't consider death as a plague, but as a fate handled by a wise deity. Take advantage of that power to use the domains of your gods to implicitly present them, it can make your story give detail about its lore in a subtle and immersive way.
