If a species is eating one specific type of food, their digestive systems have evolved to become adapted to extract the most nourishment from it and to be resistant to eventual poisons. This also likely causes their tasting buds to "like" that type of food, thus driving them instinctively towards the food they are best adapted to. One distinctive feature might be the nourishment human brains yield to them.
Plus they have the skill sets required, as described in the question, to hunt for humans, so why should they forsake their nature to go against it? Morality could be a reason, however Nietzsche may disagree:
He tells us that “man needs to supplement reality by an ideal world of
his own creation.” That is, we are compelled by our biological natures
to see the world through moral lenses, judging it in terms of good and
bad, although the world is neither in itself.
Which could mean that ghouls would not see killing and eating humans as amoral, just as we don't see much amorality in eating cattle even though we could resort to eat stuff which is already dead or about to die like fallen fruit or animal corpses. Why should ghouls eat things which are less nutritious and tasty to them, which may be the things allowing them to reach their peak healthiness?
Another factor could be danger. However, that's something all species have to deal with anyway, especially carnivores. What if they are better off if continuing to take that risk, because doing what humans do simply does not cut it well enough (at least not up to ~2 centuries ago), or if they are intellectually-culturally unable to even consider that? It's not like humans consistently enact the most moral and best systems imaginable, so why would you expect them to step over their shadows in such regards - even if they are capable of thinking as humans?
There are plenty of arguments against this notion, but relations based on violence and coercion (parasitic or exploitative) usually do not argue in fair terms, nor do they benefit from doing so. The predator will always seek for things that makes his life easier, or whatever is needed to adapt. It is likely it will never even come to that consideration.
I may have a, let's say, dark undertone in this, but I guess you are good to go with ghouls preying only on humans, not only on a biological standpoint, but also on social, moral and intercultural. Maybe it can assist as fuel for explanation from the side of ghouls.