A planet retains its atmosphere mostly by having a high enough escape velocity instead of by having a high enough surface gravity.
Thus rotating a space station to simulate surface gravity would not work to create any sort of simulated escape velocity.
A space station massive enough to have a sufficiently high natural escape velocity would be far larger than the minimum mass to pull itself into a solid and roughly spherical shape. Thus it would not be an artificial space habitat or space station, but an artificial habitable planet, with the atmosphere on the outside of a vast sphere of matter.
If the space station has a simulated surface gravity of 1 g on its inner surface, its atmosphere may behave similar to Earth's atmosphere.
In atmospheric, earth, and planetary sciences, a scale height, usually denoted by the capital letter H, is a distance (vertical or radial) over which a physical quantity decreases by a factor of e (the base of natural logarithms, approximately 2.718).
Earth's atmosphere has a scale height of about 8.5 kilometers.
Earth loses gas molecules and atoms into outer space from the exosphere, the very, very, very thin uppermost layer of Earth's atmosphere. So if the side walls of a rotating space station extended vertically up to about the distance to the bottom of Earth's exosphere, that should be high enough to keep the atmospheric loss rate about as low as on Earth.
The height of the exosphere is between 700 and 10,000 km from the earth's surface.
So the side walls of the rotating space habitat should be at least 700 kilometers tall to hold in the gases of of the thermosphere, mesosphere, stratosphere, etc. from escaping.
But if the rotating space habitat has a radius less than the radius of Earth (6,371 kilometers) the simulated gravity will decrease with height faster than the gravity of Earth decreases with height. That would mean that there would be less gravity to compress the air at a specific height. That would mean that the scale height of the habitat's atmosphere would get higher and higher with attitude and the atmosphere would loose density with height more slowly.
Thus the smaller the rotating space habitat, the higher the side walls would have to be to enclose all of the atmosphere up to the bottom of the exosphere. With a small enough rotating space habitat the thermosphere with its somewhat denser gases would extend all the way to the center of the habitat. With a smaller habitat, the mesosphere with even denser gases would extend all the way to the center of the habitat.
Thus it might be necessary to build the side walls even higher than 700 kilometers to keep atmospheric gases from escaping too fast.
If a writer plans to write a story of with a very low score on the Sliding Scale of Science Fiction Hardness:
They won't have to worry about the problems of building 700 kilometer or higher side walls to their rotating space habitat or the problems of having a rotating cylinder with a radius over 700 kilometers and simulated surface gravity of 1 g from flying apart.
If a writer wants to have a high score of the scale, then they will have to worry about constructions with forces that will exceed the capacity of the strongest known materials by many times.
Personally, I can't stand waste. So I don't like the idea of a rotating space habitat without a roof. Even if the atmosphere is lost into space no faster than Earth loses atmosphere, that is still a waste.