I'll address storage of antimatter, because that is the one thing in your question humans have done successfully so far. While we may someday build an antimatter-based propulsion device, it's a ways off. Fortunately, storing antimatter is much easier to do.
Currently, the best way to store antimatter is a Penning trap. It uses a magnetic field and an electric field to store charged particles. The reason we need both is that a magnetic field or an electric field could not keep a particle in a stable position on its own thanks to a mathematical result know as Earnshaw's theorem. Using both types of fields ends up giving us the required stability. This approach of course restricts us to using charged particles - protons and electrons, for example - but this isn't a significant problem, as these are the types of antimatter that are easiest to produce.
Unfortunately, Penning traps are used primarily to store antimatter, not to provide an annihilation chamber. When you bring the antimatter into contact with matter, you can't simply have it in the storage area. If it's in a small storage area, the energy released may destroy the Penning trap (or whatever else you're using). If it's in a large storage area, the explosion probably won't be near whichever end of the craft you designate the rear. Either way, the explosion won't be directed rearwards, as with a typical rocket.
The solution like this might be to accelerate the antimatter and matter out the end of the spaceship. Particle accelerators do this via superconducting magnets. The problem is, these accelerators are incredibly large - the Large Hadron Collider is 27 kilometers in circumference! Perhaps that would be tough to do on a small spaceship. To solve that issue, I would suggest using a small ion engine to accelerate the matter and antimatter. Have them accelerated perpendicular to each other and away from the ship, and you could direct the explosion towards the rear.
The idea isn't perfect; for instance, releasing any particles from the trap will lead to distortions in the shape of the fields, potentially breaking the structure required for stability. Since fuel will be needed on an essentially continuous basis, this could be a significant problem, as even small deviations from the desired configuration can grow over time. Fortunately, only small amounts need to be siphoned off at a time, as celtschk's answer demonstrates that fuel consumption rates are low.