In my world, I am attempting to design a primitive organism to be the ancestor of life on an artificial planet with all chemicals necessary for life. I want the organism to be as small as possible (say 50 nanometers in diameter). Is such a small size possible?
50 nanometers is awfully small. In fact, this is on the same order as many viruses, which can be 20-300 nm in size; to a first order approximation, this figure seems a good fit for Cort Ammon's estimate of 16-18 nm DNA strands. Some viruses are smaller along one dimension, but larger along others; Wikipedia mentions that some filoviruses can be 80x1400 nm in size.
In contrast, bacteria are usually in the 500-5000 nm (0.5-5.0 µm) size range. While the issue of whether a virus constitutes life is as of yet undecided, there is little doubt that bacteria can be considered to be life.
Hence, the boundary for true life, at least in an Earth-like environment, appears to fall somewhere in the 300 to 500 nm range, which is about one order of magnitude larger than what you are aiming for.
It is possible that some extreme biology and evolutionary pressure might be able to support living organisms smaller than Earth bacteria, but I find it highly doubtful that it would be able to reduce the minimum viable size of an organism by such a large fraction. There is also no real evolutionary benefit that I can see to becoming that much smaller, hence very limited evolutionary pressure in that direction. If the limiting factor is indeed DNA base pair size and the number of nucleotides needed, then you may need some completely different biochemistry to make a significantly smaller organism viable. Based on discussion in the answers to Why would life on a different planet use DNA?, it's likely that life would settle on DNA or at least RNA, so DNA strand size could be one factor establishing a lower limit for the size of life even on a completely different planet.
At the very least, again to a first order approximation, any living organism must be large enough to support some form of procreation, even if that is done by cell division. Even cell division would appear to need the cell to be well above its minimum viable size before the division. Because they co-opt existing cells, viruses do not have that size constraint, and in fact would appear to have an evolutionary advantage to being smaller (less things to go wrong leading to higher probability of success).
You may also want to compare Are there organisms with fewer than 1000 neurons? on the Biology Stack Exchange, which appears to be peripherally related.
There's some precedent for it. Spiegelman's Monster is a self-replicating strand of RNA, which they eventually got down to 48-54 nucleotides. I don't have the conversion from RNA nucleotides to length, but if I pretend that was 48-54 base pairs of DNA, we can use DNA's length of 340pm per base pair to arrive at a 16-18nm long strand.
It's an open question whether you can say much about our actual abiogenesis from studying Spiegelman's monster, but it's at least a point suggesting its not impossible.
That all depends on whether or not you consider viruses to be living things or not. Typically, viruses are far smaller than other living organisms. For example, Wikipedia tells us that
The smallest double stranded DNA viruses are the hepadnaviruses such as Hepatitis B, at 3.2 kb and 42 nm; parvoviruses have smaller capsids, at 18-26 nm, but larger genomes, at 5 kb.
If you don't consider viruses organisms, then the same Wikipedia page tells us
Prasinophyte algae of the genus Ostreococcus are the smallest free-living eukaryote. The single cell of an Ostreococcus measures only 0.8 μm across. The smallest genome of any Eukaryote is Guillardia theta with a genome size of only 551 Kilobases.
If you're looking for an actual animal-like organism, here you go (from the same Wikipedia article)
The smallest crustacean, and indeed the smallest arthropod is the tantulocarid Stygotantulus stocki, at a length of only 94 µm (0.0037 in).
Most of the size of organisms comes from their need to compete with other organisms and survive a range of hostile environments. Your ancestor, however, doesn't need to compete with anything but itself and can be seeded into the most welcoming of environments.
Evolution will handle the rest.
Consider this: the original forms of life on Earth must have been vastly smaller and simpler than today's life (due to the simple mathematics of what can plausibly have arisen by mere chance). In fact, they were likely a few molecules co-operating together (ref) and between there and modern life there must have been innumerable intermediate forms that transition the range of size between a handful of small molecules and the smallest extant cells. The minimum size of modern life is thus an extremely poor guide to the possible size of an artificially generated seed life.
One famous example of a potentially biological organism which is smaller than any of the examples listed is surprisingly Martian in origin.
The long rod-shaped structures were analysed by electron microscopy, and they found internal structures that resembled terrestrial bacterial precipitates.
While it is disputed whether these are truly biological deposits as opposed to chemically or physically formed structures, if these objects are truly Martian microorganisms, their extremely small size (20-100nm diameter) potentially fits the range of sizes specified in the question.
Similar to what @si42 said, it depends on how you define organism. You can take as example the model of abiogenesis (origin of life from non-living matter) proposed by the Nobel biologist Jack Szostak. The model was CONFIRMED to be possible and capable of originate life. That it actually did it on Earth or some other place is another question.
The model works with replicating micelles and vesicles, one of the papers says:
We tested the concentration dependence of the micelle to vesicle ratio independently by measuring the turbidity of vesicle solutions that had been extruded to 50 nm to eliminate spurious effects due to variation in vesicle size.
I did not read the entire paper, but I think that we can assume that they were of 50 nm or less.