I'm not a biochemist. Biochemistry is super-duper complicated. The following is essentially wild speculation, but hopefully it can help guide your thinking.
The overall equation for photosynthesis on earth is:
$$6 CO_2 + 6 H_2O + \gamma \rightarrow C_6H_{12}O_6 + 6O_2$$
Here are the important things to note about this process, from the perspective of changing it. These features need to be present to allow for anything like the photosynthesis we see on modern earth:
- The reaction requires energy to proceed (is endothermic), rather than releasing energy when it proceeds
- The solid output of the reaction (glucose) is a store of energy that is stable enough not to spontaneously decompose
- (A) Glucose is a moderately complicated molecule. This allows the biochemical system to manipulate it with a good degree of specificity using targeted proteins, and minimizes the chances that it will cause unwanted side reactions.
- The byproduct of the reaction is a gas, which can easily escape the plant. Liquids are OK too, but not solids (which are difficult to transport out of the plant)
You ask about instead using the reaction scheme:
$$nNH_3 + mX + \gamma \rightarrow iY + jCH_4$$
You also posit that this ammonia exists in a liquid state (analogous to water). This is your first problem: ammonia boils at -33$^{\circ}$C. This is a problem because at this temperature, about 50$^{\circ}$C colder than on earth, and as a rule of thumb every 10$^{\circ}$C difference results in a factor of 2 change in chemical reaction rates. That means that reactions on this planet will take place about 32x slower than on earth, which makes it unlikely that an endothermic reaction like this could take place.
You have a few ways around this. Perhaps your biochemistry includes a ubiquitous reaction that is quite exothermic, which is used to locally heat life enough for reactions to happen at a reasonable pace. Perhaps life only grow around geothermal hot-spots, where the temperature is locally higher and the ammonia is gaseous. Or perhaps the planet has a similar temperature to that of earth, but an atmospheric pressure about 10x higher, allowing liquid ammonia at room temperature (this likely creates its own set of problems).
Anyway, passing that on, let's see if we can come up with a moderately stable, moderately complex nitrogen compound to replace sugar. That will guide the rest of the reaction. I think an amino acid is probably a decent choice. I'll use glycine, because it's simple and this is already hard enough as-is:

Now, we have the reaction scheme
$$nNH_3 + mX + \gamma \rightarrow NH_2CH_2COOH + jCH_4$$
We can (stoichiometrically) satisfy this reaction using propinoic acid
$$NH_3 + CH_3CH_2COOH \rightarrow NH_2CH_2COOH + CH_4$$
Based on a quick look at the standard enthalpies of formation, this reaction should be endothermic*. Check. Glycine is a relatively stable solid (we produce it all the time in our bodies) that is moderately complex (complex enough to be used to build proteins at least). Check. We're consuming ammonia and producing methane, as you asked. Check.
So this is my submission for your photosynthesis. The next step is to come up with whatever kind of crazy pathway this reaction schema could possibly use. However that sort of thing is way over my head (even photosynthesis on earth is really very complicated, involving lots of electron transfer and stuff), so this is where I'll leave you.
Happy worldbuilding!
$*$ The enthalpies of formation I found are as follows (rounded quite a bit):
- Ammonia, -45 kJ/mol
- Propinoic acid, -510 kJ/mol
- Methane, -75 kJ/mol
- Glycine, 1430 kJ/mol
Thus the overall reaction has (45 + 510) < (1430 - 75) which implies it will not be spontaneous, with a net endotherm of about 800 kJ/mol. I believe this is a bit under half the endotherm of photosynthesis on Earth.