"Mission dictates gear".
This is a truism in any military organization, especially SOF. So what do SOF Soldiers wear? I ask you in return: In what situation?
If the SOF Soldier has to operate alongside GPF (that is General Purpose Forces - as opposed to Special Operations Forces), then the SOF Soldier will wear uniform and gear matching the Unit with which he's working on. Why? Because if I am an enemy sniper, and I'm out there scanning your troop formations, I'm not gonna waste my ammunition, hide site, and opportunity to kill regular Privates or Lieutenant. I'd look for those 'hmm, that dude looks special' targets like your snipers, senior leadership (Officers or NCOs), or SOF personnel. If your guys walk around in camouflage and body armor, and all of a sudden there's this group of guys walking around wearing skin-tight spandex and ninja gear, well, that fits the 'hmm, they look special' category. Believe you me, you would not be able to operate effectively if every time you leave the wire the enemy single you out as a target.
If a SOF unit operates on its own, well then the gear is determined by the need of the unit, as well as the ROE. Does the ROE state that you need to be clearly identified as a national legal combatant? Then you will be wearing combat gear with your flag. Does the ROE state that you are operating covertly/in a clandestine fashion? Then you may wear civilian clothes or sterile uniforms.
If the SOF unit is tasked to covertly infiltrate an enemy facility, then (insert whatever they're doing there), and then egress covertly, then the SOF unit might consider wearing enemy uniforms/clothing. Although, the wearing of enemy uniform means you are giving up your Geneva convention protection. They may wear local garb, pose as cops, business people, whatever gets the job done.
One thing they would not do is wear skintight spandex with ninja gear that screams to everyone, "I'm special!"
ETA: Consider that a significant amount of GPF gear today started out as SOF gear at one point of time. From armor, to weapons accessories, attachments. SOF may not look that different from GPF troops.