There are four components to consider:
Projectiles are by far the easiest to make. Assuming that the EPA was wiped out, go back to making lead projectiles. This is trivial with 1890s technology and decades or centuries after the apocalypse it is unlikely that there will be much functional kevlar armour remaining to resist them. There's a catch though - manufacturing projectiles that will not disintegrate as they leave the barrel of the latest 5.56mm assault rifles is tricky - in order to (try to) maximise the wounding potential of the round there is a ferocious turn rate from the grooves in the barrel. (Don't try firing older 5.56mm ammunition through new 5.56mm assault rifles.) 1890s technology may not be up to the job, and basic lead projectiles definitely will not work for these rifles. Read here for much more detail.
Propellant is surprisingly tricky. It's relatively easy to make gunpowder - it can obviously be made with quite basic technology - but making high quality gunpowder with each grain shaped ideally for the optimum burn rate (fast burning for pistols and shotguns, slower burning for rifles) is difficult. Low quality powder that burns too quickly will cause excessive wear or even breach explosions, while powder that burns too slowly may fail to cycle a semi-automatic or automatic weapon. (The projectile will also go either too high or too low at range.) @AlexP has correctly noted that semi-automatic weapons and their ammunition were manufactured in the 1890s, but the manufacture of the ammunition and its components was undertaken by industrial concerns, not cottage industry. Bolt / lever / pump action firearms will be more forgiving of low quality powder, so these are likely to be the firearm types in use. It is also likely that the effective range of firearms will be shorter due to the inconsistent powder making tight groupings at long ranges impossible.
Primers are relatively dangerous to manufacture - they are explosives sensitive enough to explode if struck sharply - but the techniques for manufacturing them without losing fingers / limbs / factories are easily within 1890s technology.
Cases are easily produced given a supply of brass and a lathe. This is also within 1890s technology. Cases can also be reloaded a number of times if they are recovered, much more cheaply than manufacturing new ones.
So, all of the problems are potentially solvable. However, they are only effectively solvable on an industrial scale by component type. In order to have all of the weapons and ammunition you want, you must have a separate production line for each of the following:
- .223 projectiles
- 7.62 NATO (.308) projectiles
- .44 magnum projectiles
- 9mm projectiles
- 00 buck shot (and/or slugs and/or every other shot size)
- Rifle powder
- pistol / shotgun powder
- large rifle primers
- small rifle primers
- pistol primers
- shotgun primers (unsure if pistol primers for 9mm / 44 magnum are compatible)
- .223 cases
- 7.62 NATO cases
- .44 magnum cases
- 9mm cases
- 12 gauge shotgun cases (maybe also wadding? - I've never reloaded shotgun shells)
This is a ridiculously complicated manufacturing requirement which is frankly unachievable without a civilisation which is back up and running again with lots of manufacturing capacity. It also replies on the firearms still working - eventually the pre-apocalypse firearms will wear out, and 1890s technology can not be used to manufacture spare parts for some of the firearms listed. What is far more likely is that by the time the pre-apocalypse ammunition stocks have run out there will be the following production lines:
- .45 pistol (M1911 clone or revolver)
- .45 carbine (bolt, lever or pump action, accepting the same ammunition as the pistol)
- .45 lead projectile
- pistol powder
- pistol primer
- .45 cases
It may not be exactly .45 if there are lots of pistols and/or projectile dies of a different calibre around (it could be .44 Magnum if you really want it), but it will be one pistol and one carbine with a high-ish calibre common cartridge, as was used at times in the western United States by some of the cowboys in the 19th century, for the same reasons. A pistol that can be fired rapidly at short range, a carbine that can effectively engage targets out to about 100m, both over-engineered to not explode if a "hot" load of powder slips through quality control.