Many gymnosperms have specialized reproductive structures (cones and strobili) which, in part, have functions similar to angiosperm flowers, although of course they are not homologous with flowers. Some are conspicuous and colorful, even. And there are many gymnosperms which have structures analogous (but not homologous) with angiosperm fruit.
(Note that the modern view is that extant gymnosperms, the Acrogymospermae, form form a clade; the angiosperms are thought to have evolved from a gymnosperm ancestor; so that the Gymnospermae lato sensu are a paraphyletic grade. Hat tip to Jack Aidley for pointing this out; Wikipedia agrees, quoting M.J.M. Christenhusz, J.L.Reveal, et al. "A new classification and linear sequence of extant gymnosperms", in Phytotaxa 19:55–70, 2011, doi:10.11646/phytotaxa.19.1.3.)
Flower-like structures

The female reproductive structure of the sago palm Cycas revoluta - from the Montreal Botanical Gardens. Photograph by Nadia Prigoda, available on Wikimedia under the CC BY 2.0 Generic license.

Male cone of Cycas circinalis. Photograph by Phyzome, available on Wikimedia under the CC BY-SA 3.0 Unported license.

A fleshy female cone of Ephedra intermedia, a gymnosperm shrub. Photograph from Yang Y, Wang Q, "The Earliest Fleshy Cone of Ephedra from the Early Cretaceous Yixian Formation of Northeast China" (2013), PLoS ONE, 8(1): e53652, doi:10.1371/journal.pone.0053652. Available on Wikimedia under the CC BY 4.0 license.
Fruit-like structures

Aril (pseudo-fruit) of Taxus baccata, the yew-tree, a conifer. Photograph by Didier Descouens, available on Wikimedia under the CC BY-SA 4.0 International license.
An aril (pronounced /ˈærɪl/), also called an arillus, is a specialized outgrowth from a seed that partly or completely covers the seed. [...] The aril may create a fruit-like structure, called (among other names) a false fruit. [...] Such arils are also found in a few species of gymnosperms, notably the yews and related conifers such as the lleuque and the kahikatea. Instead of the woody cone typical of most gymnosperms, the reproductive structure of the yew consists of a single seed that becomes surrounded by a fleshy, cup-like covering. This covering is derived from a highly modified cone scale. (Wikipedia, s.v. Aril)

Cycas media (a cycad) megasporophylls with nearly-mature seeds on a wild plant in north Queensland, Australia. Photograph by Tanetahi, available on Wikimedia under the CC BY-SA 4.0 International license.

Edible seeds of Aruacaria angustifolia, a conifer. Photograph by Rodrigomorante, available on Wikimedia under the CC BY 3.0 Unported license.
Conifer nuts are the edible seeds of conifers, which includes most notably pine nuts (family Pinaceae) and Araucaria nuts (family Araucariaceae). (Wikipedia, s.v. Confier nut)