Of the options you lay out, probably a black hole. Neutron stars are not pure flawless neutronium, they have a structure and require a certain minimum mass to overcome electron degeneracy pressure. Essentially they require a supernova to create, and if that's your FTL mechanism how do you leave your star system of origin without destroying it?
Black holes aren't a much more promising prospect, but at least they can be stable down to a reasonable size. Black hole formation from stellar collapse does produce black holes with a certain minimum mass, but to the best of my knowledge a spectrum of lower mass primordial black holes hasn't been ruled out.
Note on black hole evaporation: black holes do shed mass by Hawking radiation, but except for very small black holes the power output is very low. You would be better off choosing a smallish but long term stable black hole and generating your power output by feeding the black hole with accreting matter. You'll have a pretty hellish local environment in terms of high intensity radiation emitted from the black hole, although if the black hole is small enough and your mechanism is essentially a very small Dyson sphere to capture the black hole's total energy output then there may not be any far-field effects.
You could either invoke unobtainium to explain the creation of a black hole of a suitable size (though note that the red matter mechanism for turning Vulcan into a black hole in the Star Trek reboot is pretty widely mocked as silly) or simply state that small primordial black holes are actually quite common, and that your civilisation can capture them. Note that even this requires some pretty advanced sub-light speed technology! You will have competing problems: the larger the black hole, the harder it will be to capture and wrangle into position (a lunar mass black hole has a Schwarzschild radius of 0.1mm) but the smaller the black hole, the more precisely matter will have to be aimed to accrete onto it (a 100 million ton black hole has a Schwarzschild radius about the size of an atomic nucleus).
Now you have the problem of towing the very heavy, very dense black hole around with you. Or perhaps you don't - maybe the black hole is moored at a point in space in your star system, a bit like Deep Space 9, and is used as a power source by an orbiting field generator that powers an Alcubierre warp field from that point, the ship enters the warp field and travels to its destination without taking the black hole with it. That way, you only need 1 black hole per star system and you potentially have a simpler star ship design (i.e. any container capable of passing intact through the Alcubierre field). It also introduces a good reason for taking extreme care calculating the destination co-ordinates: if you travel through the field to a wrong destination you find yourself light years from anywhere with no onboard FTL ability. Unless you can travel back through the field quickly enough, you may find yourself stranded.