Radioactivity
First of all, I would like to say that this answer is not 100% science accurate as I don’t masteries all the subjects involved in it. At least it is a try and maybe the detail level needed for your story will be low enough to be very free about little imperfections and mistakes.
So, maybe your crystals could “simply” be radioactive. Your crystal would have to be made up of silica and a lot of impurities like Uranium (U), Thorium (Th) or Radium (Ra), etc. I don’t know if it will be enough for a quartz-like crystal to become radioactive but if it’s not, there is another solution:
Torbenite
The Torbenite (Cu(UO2)2(PO4)2·12 H2O) is a natural radioactive mineral with a pretty green color. I am not sure this a crystal but you have to admit that it kinda looks like what people think about when you told them "crystal".
[Source]
Ok, I have my radioactive crystal but what now ?
Generally, getting electricity through radioactivity is always a matter of heating water that will produce steam. This steam is then used to move turbines and so on… This is an over complicated way to produce electricity for your medieval world so is their another method?
According to this article, it seems that there is a way to obtain electricity from some complexes nanomaterials.
Quoting Phil McKenna (emphasis mine):
The materials they are testing would extract up to 20 times more power
from radioactive decay than thermoelectric materials, they calculate.
Tests of layered tiles of carbon nanotubes packed with gold and
surrounded by lithium hydride are under way. Radioactive particles
that slam into the gold push out a shower of high-energy electrons.
They pass through carbon nanotubes and pass into the lithium hydride
from where they move into electrodes, allowing current to flow.
“You load the material with nuclear energy and unload an electric
current,” says Popa-Simil.
Yes, but my medieval civilization doesn’t know how to design nanomaterials!
You are right. But maybe with a little interpretation of science we could archive to design such a material naturally. What we need is to combine carbon nanotubes with gold (and then to add a lithium crystal but it's not the most complicated part).
This paper, is talking about the possibility of finding single-walled carbon nanotubes (SWNT) in nature. One of their reflection is that:
The most likely growth mechanism for SWNTs in nature is the
transformation of another carbon structure, or the chemical vapour
deposition (CVD) of a carbon feedstock under appropriate conditions
(conditions similar to those of arc discharge synthesis could also
potentially be created by lightning strikes)
So, what if your civilization designed this material by mistake? An altar on the top of a mountain and an offering to some gods: a river of diamonds lying on a gold plate… Then a storm, a terrible storm and a lightning striking the altar! Under these condition, carbons atoms of diamond get reorder into nanotube shape as gold is mixt up with it…
This artifact is then set with lithium hydride and Torbenite crystal for decoration purpose and there you go: Electricity! (A small amount of course...)
Limits
As I previously said, you have to be free about some science simplifications: the altar part is far from plausible in a real world. Moreover, this lead to a single artefact so you will not be able to create electricity for everybody. Finally, we all know effects of radioactivity on human body (SPOILER : it never end up well), so people working around this artefact will have to be “replaceable” if they touch it regulaly and lick one's finger…
But of course this could lead to a sort of cult as the artefact is a gift from the gods and that it is an honor to be the one that maintain its power by “feeding” it with the green crystal…