The grooves in the Devil's Tower could actually help someone climb it, depending on how the scale of the grooves corresponded to their climbing method.
If the grooves of the tower don't help someone climb it they probably will not hinder them from climbing it either. You would need to get the opinion of an experienced climber to suggest a design of grooves that would hinder a typical climbing technique and then design the grooves in your tower to fit that design. And then the grooves might not hinder other climbing techniques.
I suggest that instead of using grooves to hinder climbers, you use lack of grooves and lack of ledges to hinder climbers. Make the walls of the tower as perfectly smooth as possible.
It should look like a lighthouse.
The walls should be made as smooth and slippery as possible. Any possible cracks between stones and bricks should be reduced in thickness so nobody could get a finger or toe in them, and then covered up with enough of a smooth, slippery covering material that nobody can tell where those cracks are underneath the smooth, slippery covering material.
If the underlying structural material is different from the smooth, slippery covering material, both should be very hard to break, so people won't be able to chop hand holds and foot holds into them, the way mountain climbers do. So the material should be like the stuff that Orthanc is made of in The Lord of the Rings as described in The Two Towers.
Don't have any doors or windows in the tower low enough for anyone to climb up any ladder they might have and then reach up to get a hand hold. So unless they have a modern vehicle with a fireman's ladder, about 20 feet up the wall should be enough to keep the windows out of reach, but to be safe there should be no windows below the one in Rapunzel's room.
The tower should be surrounded by a walkway just below Rapunzel's window. The underside of the walkway should be totally smooth without any possible hand holds, and should extend far enough beyond the tower walls that nobody could possibly hang on to the tower and reach out far enough to grab onto the edge and pull their self up, and to be extra sure it should extend considerably farther out than that.
And there should be nothing immediately below the walkway for anyone to hold on to anyway.
The walkway at the top of the tower should be high enough above the ground that nobody could bring a ladder tall enough to lean against it.
Of course, Rapunzel's hair should reach to the ground, or at least low enough to be reached by someone standing on the ground.
Normally human hair can only grow to about four feet long, and the longest human hair ever was probably about 20 to 25 feet long, which seems a little short for keeping the tower above ladder height. So possibly magic makes Rapunzel's hair several times longer than a non enchanted person could grow. But the problems of having super long hair indicate that readers would probably find hair no more than fifty feet long far believable than hair five hundred feet long.
Of course such a tower would seem to be impossible to get Rapunzel into before she is there to let down her hair.
Presumably there originally was a manually operated winch or a rope ladder to get up the tower, and once Rapunzel was inside that winch or rope ladder would have been hidden in a locked closet, preferably before Rapunzel was old enough to remember it and think about using it as a less painful way of bringing people up, or even as a way to get out of the tower. Or maybe it was thrown down outside the tower.
(one way for Rapunzel to make people climbing up her hair less painful would be to wrap it around a hook or something attached to a wall before letting down the further length of it, thus having the person's weight pull on the hook and not on her head.)
Or maybe the tower originally had a door on the ground floor and upper floor windows, and scaffolding around it. So they took Rapunzel to the upper story using the inside staircase and then sealed up the door with masonry and poured concrete through the second floor window to fill up the ground floor, then blocked up the second floor window and with masonry and poured concrete through the third floor window to fill the second floor, and so on. And then tore down the scaffold when they were finished.