You've hit on the major drawback with solar (or wind) power today: availability factor. It's difficult to predict when these will be unavailable, so you need a backup strategy. Currently, energy sources like this are supplemented in a few ways:
During peak loads or times when the sun/wind is unavailable, many utilities bring online less-renewable generators to meet demand. Typically this is Natural Gas turbines, since they have good ramp rates (they start generating quickly) and relatively cheap, easily storeable fuel. It may also be coal, hydro, or other types of plants.
Buy from other utilities
If a utility projects that they will not be able to meet the demand of their market, they have the option to buy generation on an energy market. They can also do this if electricity is cheaper to buy than generate for whatever reason. Generation is constantly being bought and sold much like stock in a stock market. In fact, many utilities need to purchase electricity at peak load times (usually 5pm) from regions to the east or west who are pre- or post-peak and have unused capacity. This is made possible by the power grid, a network of high voltage power lines for transmitting power of long distances. In an all-solar economy, this may not be possible at all hours of the night, due to losses over very long distances. You may offset those losses with extremely high voltage lines (higher voltage helps reduce losses) or by hand-waving room-temperature superconductors.
This is the most plausible place for improvements to make an all-solar grid feasible. Generally speaking, storing energy as electricity (in batteries or capacitors) is not cost-effective at the scales we're talking about. Fortunately, that energy can be converted into a form that's easier to store. Here are a few suggestions:
Use the energy to heat up a fluid, such as a molten salt. Some fluids have very good heat capacities, and can store energy effectively over night if kept in insulated tanks. When power is needed, the salt is used to heat water to produce steam and turn a turbine. This is not far from what concentrated solar plants currently use.
Use excess energy to spin a flywheel. Later, use the inertia of the flywheel to turn a generator to produce electricity.
Use excess energy to pump water up a hill to a reservoir. Later, let it flow through a turbine to produce electricity. This is the same principle that hydroelectric dams work on, although we let the water cylce do most of the pumping there. As people have pointed out in comments, there are several examples of this in use.
Pump air into a decently-sealed cavern underground. Later, release it through a turbine to generate electricity. There is a plant in the southern US that does this, and at least one more in Germany.
Use electrolysis to separate the hydrogen and oxygen in water. Later, burn the two to produce water and heat. Heat steam and turn a turbine to generate electricity.
None of these are particularly efficient, but that's not a big deal. They all use otherwise wasted energy to store a portion of that energy for later. It's difficult to make them cost-effective today, but a breakthrough on that front could make your solar-only world feasible with very near-future technology.
Currently, if a distribution utility cannot meet demand, they strategically shed load to prevent cascading failure. While less than ideal, it can be a reasonable solution, especially if your customers have access to a schedule.
Unfortunately, I'm limited to two links by my low reputation. I will add more when I'm able.