As asked in the title, what would be the upper limits to a rocky planet's size in orbit of an O type star, like 10 Lacerta's habitable zone? Density of the planet must be enough to stand on without falling through (Sorry I couldn't find a reliable source on a formula to calculate this) and mass should be high enough to retain a reasonable atmosphere.
The boundary between rocky planet and gas giant is blurry. However, it is primarily based on mass, not size. There are several categories of planets from Earth mass range up:
Terrestrial(the inner planets in our solar system, up to about 1 Earth mass).
Super Earth(what Planet 9 might be if it exists, generally 1 to 10 Earth masses).
Mini-Neptune/gas dwarf(recently discovered exoplanet class, generally up from 10 Earth masses to Neptune/Uranus mass of ~17 Earths)
Gas giant(everything from ~17 Earths up to the threshold of fusion, 13 Jupiter masses).
So what you probably would like is a mini-Neptune right on the edge but not quite a gas giant. I'm going to guess maybe 13 to 15 Earth masses. Especially if its composition is mostly lighter elements, it would be easier to form in the short lifespan of an O type star(as opposed to a super Earth which would not have time to accrete in only a few million years). Incidentally, Wikipedia says it might be approximately 3-4 times Earth radius in size, as it would be towards the upper end of that spectrum.
This does not have to be a showstopper for life though, you can still have a Sagan or Clarke like ecosystem similar to a gas giant's without a solid surface(i.e. multicellular life).
Hope this helps!
Rocky planets likely do not form around O-type stars. The lifetime of an O-type star is a few million years, which is less than the estimated timescale to form the terrestrial planets. The intense radiation of the star would tend to rapidly destroy the circumstellar disc. You may be able to form a bunch of protoplanets but it won't have enough time to sort itself out into a stable system.
You can't really talk about the habitable zone of such a star. Sure, you can find a distance where the total radiation would keep a spherical object at the right temperature for liquid water (but your actual protoplanets are still going to be hot from formation, covered in oceans of lava and blanketed in primordial atmospheres containing substantial amounts of superheated steam, vaporised rock from impactors and suchlike), but the amount of high-energy radiation put out by the star would likely lead to extreme water loss from the atmosphere for a couple of million years, until the star expands into its post-main sequence evolution and really starts to cause problems. The strong stellar winds of these stars may also be problematic for retaining an atmosphere.
The environments around O-type stars are nasty, nasty places and you don't want to be anywhere near them.