The solution isn't actually too bad; I had this as a problem in AP physics in high school. It doesn't have as much symmetry to exploit as a concentric shell, but it still has a lot of symmetry--as long as the cave and the enclosing body are both spheres, and the enclosing body has uniform density, you can treat everything as rotationally symmetric around the line connecting their centers.

From there, you can treat the cave as a body of negative mass whose gravity is added to that of the enclosing body. The somewhat surprising end result is that gravity is *constant* inside a spherical cave, and antiparallel to the offset of the cave center from the enclosing body's center.

Because of the rotational symmetry, we can reduce the problem to two dimensions to show that the field is in fact constant everywhere in the cave.

Let the enclosing body have radius $R$ and density $\rho$, the offset between centers be $o < R$, and the cave have radius $r < R - o$. The gravitational field *inside* a body of uniform density is $g \propto \rho x$. Expanded to 2 dimensions, the total gravitational force is $g \propto \rho \sqrt{x^2+y^2}$, but broken down into vector components we get $g_x \propto \rho \sqrt{x^2+y^2}\cos\theta = \rho \sqrt{x^2+y^2} \frac{x}{\sqrt{x^2+y^2}} = \rho x$, and similarly $g_y \propto \rho y$. The gravity due to the negative body that creates the cave when added to the enclosing body is $g_x \propto -\rho (x-o)$ and $g_y \propto -\rho y$. Adding these together, we get the summed components for the net gravity vector to be $g_x \propto \rho x - \rho (x - o) = \rho (x - (x - o)) = \rho o$ (i.e., a non-zero constant), and $g_y \propto \rho y - \rho y = 0$.

So, the total gravity is axis-aligned, constant, and dependent only on the density of the enclosing sphere and the eccentric distance. This is in fact the only way I know of to get an exact constant, uniform gravitational field with finite amounts of material (the infinite option being the space above an infinite plane).

To get the actual force (not just a factor it is proportional to), add a factor of $\frac{4 \pi G}{3}$ to get $g = \frac{4 \pi G}{3}\rho o$.