It might work, but at a cost
Your suggestions is to skip the synapse and have the electricity directly stimulate the next neuron. Lets modify that with "the next dendrites", as the electricity could otherwise simply go down the next neuron as well, exiting it. This is different from the existing electrical synapse, as the signal will be transformed back into chemical signals in the receiving neuron, which fires again. This would reinforce the signal, as opposed to electrical synapses where the signal will be at best the same and mostly weaker.
The problem is that the synaptic gap is a place where a lot of signal modification is happening. Hormones determine a lot in the amount of release of chemicals and the duration they're allowed to be active in the synapse. This signal modification will determine a lot. It'll help you spot/smell food earlier when hungry, or an available mate when horny, or where to find weapons or strategically better positions when in danger. So it helps determine both thought processes as well as make some signals more clear than others. You'll be missing them most likely in your setup. At best you can mimic the system inside the neuron, but that might just make the whole exercise redundant.
Finally the refraction times. You use electricity to stimuli the chemicals to set off another signal. That means that if the neuron fires, it could stimulate itself. In normal neurons this is also the case, but thanks to refraction times this can be prevented. You would have to add refraction times to the chemicals as well, to prevent self stimulation that could last past the normal refraction time.
My answer to achieve your goal, the removal/reduction of synapse time, might be more Si-Fi than you want. Yet I hope I'll make a good case.
Optical synapse
As I've tried to explain above, the synapse is still an important step in the signal process. To keep this information modification, I would propose an optic synapse. The idea is simple. instead of chemicals, light is shone via a bioluminant lamp in the synapse gap. On the other end, photo-receptors will get agitated, releasing the chemicals into the nerve cell to activate.
The gap itself can be modulated by chemicals that reduce the light, reducing the signal. The chemicals are then in turn modulated by reuptake chemicals, which are also present in a normal synapse. Hormones would affect the functionality of the bio luminescent lamp, making it activate longer or shorter. Possibly also more intensely or weaker. That way we have all normal synapse functions covered, namely duration and intensity. In the meantime we're decreasing the time of the synapse as it isn't relying on slow chemicals but fast light.
Fiber-optic nerves
But why stop there? A synaptic gap where we transform electricity into the light seems inefficient. I would skip electricity altogether and make the whole neuron a fiber-optic. The neuron will fire into a biological fiber. As long as the fiber isn't bent too much, the signal should arrive without problems nor appreciable deterioration. This will be quicker than electricity and saves the step of having the electricity transform into an optical signal. It is also quicker than electricity, a goal many here on the site want. The neuron will simply fire with a bio luminescent lamp into the fiber and it'll arrive at the other side. The fiber will terminate at the dendrite, where it'll have a small hollow with room for the modulating chemicals, making it an enclosed system.
replacing the nervous system
Will you be able to replace the whole nervous system with this? Nearly. Depending on the neuron, the axon will branch out at the last moment to several dendrites. Either the light should be enough to flood the fiber and it'll distribute itself at the branches, or there would need to be a "distribution neuron". The first neuron will fire all along the axon fiber, which will terminate at the distribution neuron. This neuron will activate, firing light over several axon fibers instead of one, ensuring every neuron will get the message. That does mean a slowdown of the signal in most cases, but with the much faster synapse still a huge net gain.
Higher efficiency
Fiber-optics have some great advantages. Information travels with the speed of light, literally. The signal is clear and won't deteriorate quickly. Many signals can be send over a signal optic.
This last one is both less impressive than you might think as well as game changing. You're not likely to send the information of 20 neurons over one line, having the distribution neuron understand what information needs to be send over what fiber and do it for you. However, you can separate the normal signals that go over the line. Neurons often pull double, if not triple or more duties. For example, although pain pathways are partly separated, they do use existing neurons to transfer information. A neuron that fires for pain can't be used for anything else at that moment, but much like a computer it can switch between the signals fast enough that you'll not notice. Still it represent a loss in signal. Fiber-optic nerves might skip that problem. They can send different light wavelengths down the optic, be received by different photo-receptors that release the chemicals so the correct wavelengths are passed on.
There is also the option to go two ways with fiber-optics. Electrical neurons can only fire one way. Light however can be send both ways at the same time without interference, allowing for further merging of neuron lines/higher density of information.
Also the refraction times. Normal electrical neurons work in pulses with a waiting period right after each pulse. This is to prevent noise as well as over stimulation of the neuron. Optical nerves might not need that. They can work on a continuous scale, or with shorter refraction times, as it's easier to start and stop the light wave. This allows again higher information density.
Other improvements are that fibers potentially use less space than electrical axons, the fiber might break and still work, and are immune to electrical interference.
This last one is important. There might be a maximum of neurons you might pack together if they're electrical. They will eventually start producing noise on each other lines. Fiber-optic nerves might prevent this fully, allowing you to stuff the brains full with nerves.
Solvable problems
Now we have a working system of optical nerves. Still there are some problems. Bending of fiber-optics isn't good for the signal and it can bounce back. Luckily the spine seems okay, but many joints in the extremities can move and make too sharp bends. To circumvent this, all neurons must have a electrical bridge between these parts. A neuron in front of the bend will fire the electricity, which will excite the lamp on the other side of the bend to fire down the fiber.
The "distribution neuron" can take up space required for neurons, making it more crowded than normal. The higher efficiency might reduce the amount of neurons needed, so this wouldn't be an issue.
In the brain the fibers might make more bends that aren't suitable for the light to travel through. That is not a problem, as the fibers are stationary. They don't have to follow the normal fiber-optic procedure and can grow in a wholly different way, simply reflecting the light around a bend. That way all neurons in the brain can be optical as well.
There is no difference with meyeline sheaths, which can help with the importance of signals. The meyeline speeds up normal electrical based neurons, making it not only faster and clearer, but also more important most of the time. This will be lost. Fortunately that is where the modulation can come in handy. They can have brighter lights and with all other advantages it shouldn't be a problem to identify important stuff still.
Potential problems
Although the above sounds very nice in theory, practice might be quite difficult. The bioluminescence as well as the photo-receptors might cost a lot of energy. Despite it seeming cool in nature, constant use might still warm up neurons just enough that it'll interfere with enzymes for example. The light given off might also be too little to be reliably picked up by photo-receptors. They might never receive enough light to give off enough chemicals to stimulate the next neuron.
Conclusion
If it works, it would reduce the normal synapse to much less than half or the original. One side of of the synapse is skipped altogether, as well as instead of chemicals slowly moving over the gap it'll go with the speed of light. The only slowing down of a signal is now in the translation of the light signal into chemicals to stimulate the next neuron. Moreover, the nerves themselves are also sped up as they use light instead of lightning. There is a potential for a higher density of information, both ways on a neuron pair, as well as on multiple wavelengths at a time. Of course it's a theoretical idea, but so is electrically stimulating the next neuron.
