There are plenty of materials in the atmosphere
Jupiter's atmosphere has plenty of chemical constituents that would be useful in making a whole variety of solid objects. Commonly occuring materials in the atmosphere include methane, water, hydrogen sulfide, ammonia, and phosphine. This give you (the Jovian cloud alien, JCA) access to plenty of simple elements like Oxygen, Carbon, Nitrogen, Sulfur, and Phosphorus, in addition to the omni-present Hydrogen and mostly chemically useless Helium.
In general, proposing the technological advancement of a gaseous being is a difficult task. So many concepts would be so radically different from our own, it is hard to make any comparisons. So I won't suggest at all how these JCA might have developed the requisite chemical knowledge to manipulate these materials. But they are present, nonetheless, and could be used.
With Carbon, Nitrogen, Hydrogen, and Oxygen, all of the organic polymers reguarly seen on Earth could be fabricated, so all sorts of plastics and aramids are possible, like Kevlar or even carbon nanotubes. Most of the normal rocket propellants could also be made from these same sources. The biggest weakness for space applications that I can see from cursory research is that without larger elements it is difficulty to get the density to make good heat shields. You would probably have to come up with some solution using the phase transitions (solid->liquid->vapor) of large, dense sulfur or phosphorus based polymers, to get good energy dissipation per unit volume.
Given the cool temperatures in the upper parts of Jupiter's atmosphere, it wouldn't be hard to support solid objects (like a rocket or space probe) on a hot-air balloon of some sort, and again there are plenty of available materials to make a balloon shell out of.
All and all, I'd say the answer is: there are a lot of ways this is possible.