Same way it was done on Earth: timekeeping.
Assuming the planet rotates, the tower, being at the pole, will rotate with the same angular velocity. Arbitrarily pick one place that will be the designated Prime Meridian. We'll call it Grenich. That will be 0 longitude. Now use a reliable clock set so that when the sun is at the highest point overhead at Grenich, it is the midpoint of the day, or start of the day, or however you arrange your timekeeping.
Have a portable second clock that keeps the same time, and travel spinward or anti-spinward from Grenich. If you want to know what longitude you are at, take a sighting on the sun when it's at the highest point in the sky (local noon) and compare it to what the time reads on your clock. The difference between noon where you are and noon in Grenich (which you know, because the clock is set based on it) can be used to determine calculate your latitude.
Simple example using Earth: Earth rotates 360 degrees in 24 hours, so 15 degrees an hour. When you determine local noon, it's when the clock, set at Greenwich Mean Time, reads 11:30am. So you've reached local noon a half hour before, so you know you are about 7.5 degrees spinward (East) of Greenwich, thus your longitude is 7.5 East. If your clock reads 2:45 pm, you are 2.75 hours behind Greenwich, thus around 41.25 (or 41 deg 15') West.
Note that this method works on Earth even in polar regions in 24 hour daylight because the sun still reaches the highest point in the sky at noon local time. In your setup, it's even easier because there will never be 24 hour sunlight except on the top of the cylinder.
Now, that suffices for low-tech. There are a couple of options for more high-tech solutions. One that comes to mind is that the civilization sets up radio transmitters at regular intervals around the cylinder, one high up, one low down, with paired transmitters at each interval. Let's say they set up 36, so one pair every 10 degrees, and each broadcasts a simple signal, say its longitude (from 0 to 350) plus N or S to indicate if it's in the north or south.
Your "compass" would consist of a radio direction finder that can measure the angle between two or more radio signals. You would first look for the signals the closest to directly north and south of you. Let's say it's the 30N and 30S signals. You then measure the angle between the transmitters. If they are 180 degrees apart, you are directly on 30 longitude. If they are closer than 180 degrees, basic trigonometry will tell you how far east or west of 30 degrees longitude you are. If you wanted to get fancy, it might lock on the 20N/20S or 40N/40S transmitters to make a second reading to verify.
If you can't get evenly space paired intervals, the system still works, the math is just slightly more advanced: you might be reading off the 33N signal and the 38S one, for instance.