There two ways of making a planet inescapable. Assuming the advanced aliens have been removed from space and now are all living on their native planet. For maximum effectiveness both methods can be deployed simultaneously.
Low orbital space around the planet is filled with a high density of objects. This will be old satellites, space junk, derelict space stations, old boosters, disused equipment, and wrecked spacecraft left over from the last galactic war. Once critical density achieved any collision between any two objects will produce debris with a high probability of causing further collisions. This is an ablation cascade or as it is known the Kessler Effect. Once the Kessler cascade has reached a sufficiently high level this will make space travel if not impossible then extremely dangerous and difficult. Certainly for spaceships trying to pass through the planet's orbital space.
The Kessler cascade will have to be keep supplied with additional space junk and debris to maintain a lethal zone around the planet. Otherwise the ablation cascade will dissipate over centuries.
If the planet is home to an advanced sapient species it is safe to assume that it has a strong geomagnetic field. This means the planet will have its own system of Van Allen radiation belts. The second method simply involves increasing the strength of the radiation to a sufficiently high level that any living creature, that is to say, one of the advanced aliens, will be exposed to a fatal dose of radiation when they attempt to escape from their own native planet.
The sapients responsible for trapping the advanced aliens on their own planet, will use a form of star lifting technology to direct an increased solar wind and coronal mass ejections towards the planet. The energetic particles will be trapped by the planet's geomagnetic field where they will oscillate back and forth in the radiation belts.
There are two methods of star lifting technology that could achieve this outcome. They are thermal-driven outflow and the so-called "Huff-n-Puff" technologies. The key to both of them is they use magnetic fields to direct plasma and particles towards the planet.
Together the ablation cascade and stronger radiation belts will help imprison the advanced aliens on their planet. However, just in case the aliens can evade these barriers a fleet of drone spacecraft and warships can be kept in readiness to intercept any escaping spaceships. Local asteroids and moons can be armed with railguns, missile batteries and laser arrays.
The entire system can be run and maintained by an army of Von Neumann constructors. While a typical Von Neumann machine is self-replicating system that will keep on making more copies of itself. The Von Neumann constructors will, yes, more copies of themselves, but they will be programmed to make more of the systems needed to keep the ablation cascade at its full strength by continually adding more objects and material to the debris clouds, and the star lifting systems to further add plasma to the radiation belts. This is in addition to building more drone spaceships and weapons for use against the aliens on the planet.
Together all these systems and lethal physical barriers will make the planet inescapable.