In practice it can't be done, but that's no fun. Flowers are highly seasonal and the ones that open or close at different times of day are fairly approximate time keepers. Flower clocks, while they do exist, rarely go beyond having a circular flower bed with a clock mechanism

or sundial built in. Such as this one at the botanical gardens (Łódź)

However:
2 a.m.:
Common Morning Glory (opens)
Night-Blooming Cereus (closes)
3 a.m.
Imperial Morning Glory (opens 3-4 a.m.)
4 a.m.
Yellow Hawkweed (opens)
Dogrose, Chicory, Yellow Goats-Beard (opens 4-5 a.m.)
5 a.m.
Buttercups, Poppy (opens)
Dandelion, Morning Glories, Wild Roses (opens 5-6 a.m.)
6 a.m.
Spotted Cat's Ear (opens)
Flax, Potatoes (open 6-7 a.m.)
7 a.m.
African Marigold, Lettuce, White Water Lily ( opens)
8 a.m.
Mouse-Ear Hawkweed, Scarlet Pimpernel (opens)
African Daisies, Nolana (open 8-9 a.m.)
Dandelion (closes 8-9 a.m.)
9 a.m.
Calendula (Field marigold), Catchfly (opens)
Coltsfoot, Gentians, Sandworts (opens 9-10 am)
Prickly Sow Thistle (closes)
10 a.m.
Common Nipplewort (closes)
Star-of-Bethlehem (opens 10-11 a.m.)
California Poppies (open 10a.m.-1 p.m. only in sunlight)
11 a.m.
Star-of-Bethlehem (opens)
Noon
Goatsbeard, Blue Passion flower (opens)
Morning Glories (closes)
1 p.m.
Carnation (opens)
Childing Pink (closes)
2 p.m.
Afternoon Squill (opens)
Scarlet Pimpernel, Water Lily (closes)
Chicory, Dandelion, Poppy, Potatoes, Sandworts (closes 2-3 p.m.)
Dandelion (closes 2-5 p.m.)
3 p.m.
Hawkbit (closes)
Calendula, Spider plant (closes 3-4 p.m.)
4 p.m.
Purple Hawkweed (opens 4 p.m.)
Four O' Clocks(opens 4-7 p.m.)
Small bindweed, Allyssum (closes)
California Poppies, Cat's Ear (closes 4-5 p.m.)
5 p.m.
Night-Flowering Catchfly (opens 5-6 p.m.)
Chicory, White Water Lily (closes)
Coltsfoot (closes 5-6 p.m.)
6 p.m.
Showy Evening primrose, Goatsbeard, Moonflowers (opens)
White water lily (closes 6-7 p.m.)
7 p.m.
White campion (opens)
Fig-marigold (opens 7-8 p.m.)
Iceland poppy (closes 7 p.m.)
Daylily, Dogrose (closes 7-8 p.m.)
8 p.m.
Night flowering cereus (opens 8-10 p.m.)
Catchfly, Dandelions, Daylilies (closes 8-9 p.m.)
9 p.m.
Flowering Tobacco (opens 9-10 p.m.)
I wouldn't set my clock by it, but it would make a good centrepiece for your village. It even includes a water feature. If your village is happy to move to a slightly casual version of local solar time, then it would work nicely. There's no real reason why not beyond its hopeless inaccuracy.
Someone has actually built one, note the small water feature as promised.

'It's not very accurate in the small hours. There aren't too many night-blooming plants that grow well up here. They open for the moths, you know-'
'It's how time wants to be measured', said Lobsang
Thief of Time: Terry Pratchett