Part One of Five: Spells to See Stars in the Day Sky.
As the planet orbits its star (if your world is a planet orbiting a star) the stars visible during the night will slowly shift to the day side, and stars on the day side will slowly shift to the night side in a seasonal cycle.
Someone who uses a star for magic during the night will find it gradually appearing less and less during the night. Thus they would have incentives to make spells enabling them to see that star during the daytime, perhaps by enabling them to see wavelengths of light that penetrate clouds easier than visible light does.
And thus that type of spell might enable someone to see their magic stars during the day and the night. But no star will be visible above the horizon, in day or in night, more than about half the time if your world is a rotating sphere. So people will need to have magic links to at least two stars on directly opposite sides of the sky, and preferably more stars spaced closer together.
Part Two: A World With a Less Dense Atmosphere.
You could create a world with a less dense atmosphere but a higher concentration of oxygen in that atmosphere so people have enough oxygen to breath.
The Earth's atmosphere is nitrogen 78.084 percent, oxygen 20.0946 percent, argon 0.934 percent, carbon dioxide 0.041361 percent, and lesser trace gases, as well as 0 to 3 percent water vapor.
Humans need the oxygen to breathe, and some carbon dioxide for plant photosynthesis, and some water vapor for rain, and some nitrogen for plants to acquire from the atmosphere. The total atmospheric pressure could probably be as low as 50 or 25 percent of its current value for humans to survive, especially if one of the things they use star magic for is helping their breathing.
Habitable Planets For Man, Stephen H. Dole, 1964, discusses habitable atmospheric pressure on pages 13 to 19.
To summarize then, the atmosphere of a habitable planet must contain oxygen with an inspired partial pressure between 60 and 400 millimeters of mercury and carbon dioxide with a partial pressure roughly between 0.05 and 7 millimeters of mercury. In addition, the partial pressures of the inert gases must be below certain specified limits and other toxic gases must not be present in more than trace amounts. Some nitrogen must be present so that nitrogen in its combined form can find its way into plants.
https://www.rand.org/content/dam/rand/pubs/commercial_books/2007/RAND_CB179-1.pdf
Since the sea level atmospheric pressure of Earth is about 760 millimeters of mercury, and an atmospheric pressure of only 70 or 80 millimeters of mercury would probably be acceptable if of the right composition, your human like characters could probably survive in a world with an atmospheric density as low as 9.2 to 10.52 percent of Earth's sea level pressure.
That should reduce how much the atmosphere dims the light of the stars.
Part Three: A Nearby Star Cluster.
Make a lot of bright stars much closer to your planet than is the case in Earth's part of our galaxy.
One way to do that is to make your planet and its solar system happen to be passing though an open star cluster which has a number of hot young stars of high luminosity. Note that those hot young high luminosity stars will have different spectral types than any star expected to have habitable planets, and thus will have higher proportions of ultraviolet radiation than the star of the planet would. Possibly ultraviolet light is useful for magic.
Or possibly the solar system is passing near to such an open star cluster, and thus those super bright stars visible during the daytime may be seen from only one direction.
If the direction to the star cluster is near the plane that the planet orbits in, most of the stars in the cluster will only be visible during the night time during part of the year, but the super bright stars in the cluster will be visible day or night, but only during about half of any daily cycle of night and day. In the other half of the daily cycle someone's location will be pointed away from the cluster and the cluster will be below the horizon.
If the direction to the star cluster is close to the direction of one of the planet's poles, the stars in the cluster will always be above the horizon, day or night, in that hemisphere of the planet, circling around the pole. The brighter ones will be visible during the day and the night, and dimmer ones will be visible only during the night.
And on the other hemisphere of the planet, near the other pole, the stars in the cluster will never be visible, day or night, and star magic using the cluster stars will be impossible there. Star magic using other background stars will be possible in that hemisphere during the night when they are visible.
Part Four: A Multiple Star System.
Your planet could be in a double or multiple star system. In that case its own star that it obits will appear as a disc in the sky and seem to be a sun. One or more of the other stars in the system might possibly appear as discs also. But some or all of the other stars in the system may be far enough away to appear a dots of light, and yet many times brighter than any star in Earth's sky, so they would be visible during the day (when above the horizon and not hidden by clouds).
So you should study multiple star systems and perhaps come up with a planet orbiting only one star in a 5 or 6 star system, with the other stars far enough away to appear as dots of light, but bright enough to be visible during the day.
https://en.wikipedia.org/wiki/Star_system#:~:text=Multiple%2Dstar%20systems%20are%20called,or%20septenary%20with%20seven%20stars.
Part Five: A Fantasy World.
Of course your setting might be a fantasy world flat earth surrounded by a geocentric system, in which case the gods might have built their celestial sphere closer to the flat Earth and made the star lamps many times brighter, so that they could be seen even during the daylight on the flat Earth.