Yes, but to do it with a single body in a way that avoided deadly tidal forces, you'd need a black hole with around 10,000 times the mass of the Sun or more, and it would probably have to be orbiting a much larger "supermassive black hole"
First of all, it should be noted that all the theories of gravity that physicists use (both Newtonian gravity and the more accurate theory of general relativity) are time-symmetric, which as discussed in this article means that if you take a movie of some bodies acting under the influence of mutual gravity and run it backwards, a physicist will have no way of telling whether the movie is playing backwards or forwards (or equivalently, time-symmetry implies that it's possible to set up a second system with different initial conditions such that if you calculate the behavior as time runs forwards, using the same laws of gravity, this second system's dynamics running forwards in time will look just like the first system's dynamics played backwards). Some reversed movies might be more unlikely than the forwards version if the forwards version featured a significant increase in entropy, but this would usually require a large number of different objects (like a collection of dust particles collapsing inwards due to mutual gravity), if you're just dealing with two bodies that don't crash into each other or otherwise break apart the change in entropy will probably be small. So, if you can come up with a situation where an object is initially traveling at only a small fraction of light speed relative to a larger body, but uses a gravitational slingshot to increase its velocity relative to the larger body to a much larger fraction of the speed of light, then the reverse scenario should be equally possible.
And the question of using a gravitational slingshot to attain a large fraction of light speed relative to the body used in the slingshot is addressed in the book The Science of Interstellar by gravitational physicist Kip Thorne, who consulted on the movie. In chapter 7, "Gravitational Slingshots", he notes that the ship in the movie (the 'Ranger') did not have sufficiently powerful rockets to accelerate to significant fractions of the speed of light on their own, but that
Fortunately, Nature provides a way to achieve the huge speed changes, c/3, required in Interstellar: gravitational slingshots around black holes far smaller than Gargantua.
Gargantua was the supermassive black hole in the movie (it was supposed to have a mass 100 million times that of the Sun), but Thorne writes that he imagined smaller black holes orbiting Gargantua. He also notes that while a neutron star or stellar mass black hole could possibly provide the required velocity change, doing so would require getting so close to them that the so-called tidal forces--the stretching people would feel due to the gravitational pull being noticeably stronger on the side of their bodies closer to the center of the neutron star or black hole than the side that was just a bit farther--would be deadly for bodies of this mass, so that a much more massive intermediate mass black hole would be needed to avoid being ripped apart by tidal forces (what astrophysicists colorfully refer to as spaghettification).
To change velocities by as much as c/3 or c/4, the Ranger must come close enough to the small black hole and neutron star to feel their intense gravity. At those close distances, if the deflector is a neutron star or is a black hole with a radius less than 10,000 kilometers, the human and Ranger will be torn apart by tidal forces (Chapter 4). For the Ranger and humans to survive, the deflector must be a black hole at least 10,000 kilometers in size (about the size of the Earth).
Now, black holes that size do occur in Nature. They are called intermediate-mass black holes, or IMBHs, and despite their big size, they are tiny compared to Gargantua: ten thousand times smaller.
He also mentions that "A 10,000-kilometer IMBH weighs about 10,000 solar masses", so that would be around the lower mass limit of what could be used to get a change in velocity of c/3 or c/4 without being torn apart by tidal forces, if you only need a change of c/10 it might be a bit smaller but my guess is it wouldn't be more than an order of magnitude.
@Aron also makes an excellent point in a comment on the answer by @AndyD273 -- namely, that gravitational assists can't actually provide a long-term boost in a ship's velocity in the rest frame of the massive body a ship gets the assist from, the velocity boost will only be seen in some other reference frame. The reason for this is that the total energy of the ship in the body's rest frame is just the sum of its potential and kinetic energy, and when the ship is some large distance D away from the body before passing close and getting an assist, its potential energy will be exactly the same as when it is the same distance D away from the body after the assist, so its kinetic energy must be the same too. Thus, a gravitational assist will only boost a ship's velocity in some reference frame where the massive body itself has a large velocity, like boosting one's velocity in the Sun's reference frame by passing close to Jupiter. In The Science of Interstellar Thorne was assuming the IMBHs were in orbit around the supermassive black hole Gargantua, and orbits around a fast-rotating supermassive black hole can reach substantial fractions of light speed, see my answer here for details. So if you want to decelerate relative to the galaxy, and your initial velocity relative to the galaxy is 0.1c, you'll probably need to find an IMBH that is itself moving at somewhere close to 0.1c (or greater) relative to the galaxy, with the most plausible astrophysical scenario for this being an IMBH orbiting a supermassive black hole.