TL;DR: probably not.
In such a world that no longer completes a rotation on it's own axis (apart from that induced by being tidally locked rotation), is it possible to have a partial turn, oscillating back and forth on it's own axis?
Nope. This would involve slowing the planet's rotation to a stop, and speeding it back up again. There's a lot of energy stored up in a rotating planet, and trying to stop it promptly will involve some Really Interesting things happening, and those are the sorts of things that you want to be viewing from high orbit.
I picture the characters view from their location as being able to watch the sun come up, rise to about midday, then appear to rotate back towards the direction of "sunrise", to become sunset.
I believe it is possible to get this sort of effect over the length of a day on a tidally locked (or near tidally locked) world, due to the phenomenon of libration
There's a nice gif of this effect on wikipedia, showing libration of the moon as seen from Earth:

You can see an apparent rocking back-and forth and "breathing" motion... the motion isn't the moon really rocking, but slight tilts and eccentricities in the orbit mean that during rotation slightly different parts of the surface are visible from Earth. The reverse is also true.
On the moon this would manifest on certain portions of the surface (around the terminator) as the Earth rising above the horizon, describing part of an analemma which might indeed involve an apparent backwards movement at some periods, from some positions.
Is there a way for this to take days instead of months in the process, but not sure what to call this type of action. I am not sure how fast I could have this oscillation occur, either. I picture an Earth-size rocky world as the planet in question.
Unfortunately the libration effects occur over an entire orbit, ie. over the whole year. On the moon they take about a month. On a gas giant moon the might occur over a few days... there are obviously other issues with gas giant moons, and by the time your peeps visit them they'll be long tidally locked. There are plenty of other questions and answers on here about exomoons so I won't duplicate them.
If you do go thus route, consider that it isn't necessarily stable for the long term as tidal effects will act to circularise the orbit which will reduce libration effects. This is important for gas giant moons... in order to induce a reasonable eccentricity for interesting libration effects you'll need to arrange an orbital resonance which has its own problems as Io's exciting geology will show.
If you had a smaller, cooler primary you might be able to find just the right combination of habitable zone and tidal-locking timescales for your needs. Suitable large outer planets might be able to give the orbit of your world the eccentricity it needs, but there's a risk of ending up with a spin-orbit resonance of the sort that Mercury has... the day length is 2/3rds of the year length, and the situation is reasonably stable. I'm not sure what libration would look like there.
There's a good chance you're out of luck, but the situation isn't so implausible that you couldn't handwave it in.