How to support population?
Assuming these refugees are untrained peasants fleeing an enemy army, the biggest problems I can see with using a natural cave are ventilation and stability. A badly ventilated cave that is starting to fill to capacity will push some people farther and farther back into the cave tunnels. Caves are dark, moist, and have a tendency to have shard drop offs and extremely narrow passages. This combined with the waste of possibly thousands of people is going to make this camp even better at transmitting disease and causing death than the average medieval town.
Without knowing how many caverns there are, how they are distributed and what their width is I really cannot calculate the number of people it can hold. I can give you some basic ideas to consider. The average worldwide height seems to be 160 - 175cm which is about 2 meters. The average width seems to be 40 to 43 centimeters or half a meter.
If each cavern is 6 meters high and (let's assume) 100 meters in width, we can fit a rough maximum of 200 people in a cavern (without stacking them). These are not ideal/healthy conditions, so let's give each person 30 meters of space. This equates to 3 healthy people per cavern, with the possibility to move this up to 6 people per cavern assuming two people per 30 meters.
Multiply this by the number of caverns and you have the rough sustainable population of the "city" assuming you can provide resources. You would also have to organize "waste teams" to haul human waste to the surface and dump it away from the city (and preferably the river since you don't want to be chugging piss-contaminated water). This job would be undertaken by the majority of people. Anyone with special knowledge (e.g. Magic users, soldiers, weapon smiths, wood workers) would be assigned to teams that utilize that knowledge. This maximizes productivity and will be improved on over time.
Magic can be used to drill ventilation shafts.
How to protect against enemies?
This depends on several factors. What is the expected size of an attacking force? If, say, the cave has a population of 200 peasants and the attackers number 10,000 knights then you are screwed. If there are a thousand peasants, 500 former feudal levies and 500 smiths, you stand a chance in a complex cave network. The best weapon you can make: a spear! Spears are easy to use. Put your carpenters to work crafting wooden poles while your stone workers and smiths create stone and metal spear ends. This way, you can arm your entire population with an efficient, easy to use weapon.
You could also use a bow and arrow (simple bows are easy to make. It's not like you need a longbow or compound bow specifically. Just something that can fire arrows!) and crowd your would be archers into narrow tunnels. Any enemy who tries to get to them will get a face full of arrows and spears. The goal will not be to stop the enemy from entering the cave, but to spend your time (before the battle) exploring and learning every cranny and nook to hide in. This way you can conduct guerrilla warfare in the cave. Make enemy casualties become soon high that they do not see any worth in pursuing your peasant army farther.