Yes
In fact there are visuses that do pass from one instruction set to another. The most famous was the ones designed to attack Iranian centrifuges. They passed from one computer to another through a variety of infection methods (USB, network), then found the controllers for centrifuges (fpgas if I remmeber right) and infected them. Now it acted different on different systems, but that is because it was designed to do so; the goal was to break the centrifuges, and spread on the conventional computers.
It being "in binary" is noise; all computer files are binary. You can interpret a file in various ways, but they are still binary. You can write a word doc file "in binary". Now, doing so without the help of tools is hard, but you said possible.
The first step is to discard the idea that a program is a specific executable. A program is a bunch of steps. The virus I imagine would run different executables on different architectures. Once it had infected a system, it would transfer a payload of how to infect other systems with different architectures.
The general way worms spread today is by probing systems to infect, fingerprinting what that system is running, then attempting remote penetration exploits that could work on the target architecture. Once they have some penetration, they fingerprint the system from the inside (more detail) and use exploits to upgrade their access (if needed), often patch the exploits they used to get in (they don't want to share), and either spread more or "call home" for instructions (join a hive).
More architectures just means more effort on the part of the worm maker. Worm makers are lazy; they just want a bunch of systems. So they find the lowest hanging fruit (common, low security systems that have what they want).
In the case of the crackers that wanted to attack the centrifuges, the centrifuges where isolated from the internet. So they had to create a package that could handle more than one target; one to spread as a worm, and one to damage as a payload.
Note, however, that the program being written "in binary" does not require that defeating it uses the same strategy. Something written "in binary" is like saying something is "hand crafted". If you have a hand crafted gun, a mass-manufactured suit of body armor can stop its bullets, and a mass-manufactured metal detector will get set off by it.
How you create something does not change what it is. All computer storage in this day and age is binary -- 0s and 1s -- so every executable, web page, script and document is "binary". They where almost all written using tools, because we have amazing tools to create files on a computer; compilers, text editors, IDEs, HTML editing suites, Typescript transpilers, office suites.
Someone manually fiddling bits (writing in binary; and how manual? Presumably he's using a tool) isn't producing anything fundamentally different than someone using tools to do the same.
