Don't worry so much about what terrestrial insects look like. You're "evolving" something from those critters. So, let's look at the first problem: what do you need for bipedal locomotion?
You need something that provides the "gyroscope" that keeps the critter upright. For humans, that's a combination of...
Our inner ears (air pressure),
Sight (visual alignment with surroundings),
Touch (heat and air motion),
and a host of other systems all "detecting" how "upright" we are (based on the brain's "I want to do this" control). The result is muscle changes that put us in the position we want (e.g., "upright").
So, first and foremost you need to think abut how your target insect would evolve those basic systems (don't worry to much about the details, you're looking for "good enough" so readers can suspend their disbelief... not so PhD's can nod their heads in agreement that you correctly predicted a million years of evolution).
Goal: how would the insect create it's internal "gyroscope?" If it still has its exoskeleton, then it will need to compensate for the loss of touch (but, for example, it may compensate by developing the ability to detect minute stress against joint muscles).
Delicate Muscle Control
Without electronic help, we really can't detect the bazillion of muscule corrections that occur moment-by-moment to keep us in the position we want (well, when you're standing on your toes trying to screw in a lightbulb with the tips of your fingers you can tell...). However, those adjustments are needed. The basic problem with insects is that most of those limbs have very little in the way of muscles (or whatever squishy goo they use to create basic contractions). They may evolve muscles/squishy-goo such that they can, but that means...
Here's the next problem. Balance requires the bendy parts to adjust in a lot of different directions. If you think about it, your toes, ankles, knees, hips, spine, and shoulders must all make minute adjustments to keep you upright. Nearly everything on those darn bugs is ramrod ridged. Now, exoskeletons means no internal bones. That's OK (...ish) if we evolve really tough "skin" that can better articulate the joints and allow for greater range (and more subtle range) of movement.
Perhaps your biggest challenge here will be side-to-side corrections. Imagine (probably because I saw the picture in another post) a grasshopper standing up. The ability of its legs to "rotate" forward (and, with evolution, backward) covers half the problem, but the basic design means very limited side-to-side correction. If you pushed on his "shoulder," he might just fall over because he can't do anything about it. Without an internal bone structure for hips, our friend might need a short but tough "tube" that extends from the abdomen to the leg, giving it a bit more separation from the body and therefore greater side-to-side range of motion.
The "Other Side" Muscles
Another thing your bugs will need is muscles on the "other side" of the "bone." Humans have muscles, for example, on the front and back of the leg. Those muscles work together to keep you upright. They also allow you to move both backwards and frontwards.
Insects can kinda do this (beetles and spiders better than most, grasshoppers would suck at it) because the legs tend to be engineered for just one or two motions. However, if we think of the "squishy goo" as hydraulics, then everything you need is basically already there: you just need a better pumping system that can precisely forward and reverse the fluid, so to speak.
Goals: joints and "muscles" (aka, hydraulics, I kinda like that idea).
Give your insect these abilities and, while the cute critter won't be the best dancer in the world, he/she would be capable of bipedal locomotion.