My research tells me that a moon around a gas-giant is not likely to be larger than 1:10,000th of the mass of its parent.
The theoretical mass limit between a planet and a brown dwarf is about 13 Jupiter masses, or about 4,131.4 times the mass of Earth. Thus if a moon can be no more than 0.0001 times as massive as a gas giant, it can have no more than 0.41314 times the mass of Earth.
Jupiter has a mass 317.8 of Earth. Its most massive moon, Ganymede, has a mass of 0.025 of Earth. Thus the mass of Jupiter is 12,712 times the mass of its most massive moon.
Saturn has a mass 95.159 of Earth. Its most massive moon, Titan, has amass of 0.0225 Earth. Thus the mass of Saturn is 4,229.28 times the mass of its most massive moon.
Uranus has a mass 14.536 of Earth. Its most massive moon, Titania, has mass 0.0005908 of Earth. Thus the mass of Uranus is 44,603.926 times the mass of its most massive moon.
Neptune has mass of 17.147 of Earth. Its most massive moon, Triton, has mass 0.00359 of Earth. Thus the mass of Neptune is 4,776.3231 times the mass of its most massive moon.
So according to the examples of gas giant planets in our solar system, a moon with the mass of the Earth could orbit around a gas giant planet with a mass of 4,229.28 or 4,776.3231 times the mass of Earth, which would be 13.307992 or 15.029336 times the mass of Jupiter. That would be a little bit above the theoretical lower mass limit for a brown dwarf.
The largest and most massive moon in the Solar System, Ganymede, has a radius of only≈0.4R⊕ (R⊕ being the radius of Earth) and a mass of≈0.025M⊕. The question as to whether much more massive moons could have formed around extrasolar planets is an active area of research. Canup and Ward (2006) showed that moons formed in the circumplanetary disk of giant planets have masses ≲10−4 times that of the planet's mass.
Canup R.M. Ward W.R. A common mass scaling for satellite systems of gaseous planets. Nature. 2006;441:834–839. [PubMed]
Mass-constrained in situ formation becomes critical for exomoons around planets in the IHZ of low-mass stars because of the observational lack of such giant planets. An excellent study on the formation of the Jupiter and the Saturn satellite systems is given by Sasaki et al. (2010), who showed that moons of sizes similar to Io, Europa, Ganymede, Callisto, and Titan should build up around most gas giants. What is more, according to their Fig. 5 and private communication with Takanori Sasaki, formation of Mars- or even Earth-mass moons around giant planets is possible. Depending on whether or not a planet accretes enough mass to open up a gap in the protostellar disk, these satellite systems will likely be multiple and resonant (as in the case of Jupiter) or contain only one major moon (see Saturn). Ogihara and Ida (2012) extended these studies to explain the compositional gradient of the jovian satellites. Their results explain why moons rich in water are farther away from their giant host planet and imply that capture in 2:1 orbital resonances should be common.
Ways to circumvent the impasse of insufficient satellite mass are the gravitational capture of massive moons (Debes and Sigurdsson, 2007; Porter and Grundy, 2011; Quarles et al., 2012), which seems to have worked for Triton around Neptune (Goldreich et al., 1989; Agnor and Hamilton, 2006); the capture of Trojans (Eberle et al., 2011); gas drag in primordial circumplanetary envelopes (Pollack et al., 1979); pull-down capture trapping temporary satellites or bodies near the Lagrangian points into stable orbits (Heppenheimer and Porco, 1977; Jewitt and Haghighipour, 2007); the coalescence of moons (Mosqueira and Estrada, 2003); and impacts on terrestrial planets (Canup, 2004; Withers and Barnes, 2010; Elser et al., 2011). Such moons would correspond to the irregular satellites in the Solar System, as opposed to regular satellites that form in situ. Irregular satellites often follow distant, inclined, and often eccentric or even retrograde orbits about their planet (Carruba et al., 2002). For now, we assume that Earth-mass extrasolar moons—be they regular or irregular—exist.
Sasaki T. Stewart G.R. Ida S. Origin of the different architectures of the jovian saturnian satellite systems. Astrophys J. 2010;714:1052–1064.
Ogihara M. Ida S. N-body simulations of satellite formation around giant planets: origin of orbital configuration of the Galilean moons. Astrophys J. 2012;753 doi: 10.1088/0004-637X/753/1/60.
Triton has a mass 2.0936 times as great as a moon formed in the circumplanetary disc of Neptune should have according to Canup and Ward. Triton is believed to have been captured by Neptune.
Titan has a mass 2.3644 times as great as a moon formed in the circumplanetary disc of Saturn should have according to Canup and Ward. Thus Titan should have acquired its mass by one or more of the processes suggested to enable moons to exceed the mass limit postulated by Canup and Ward.
But why are gas giants and their moons the only models for the satellite systems of gas giants?
The Earth is 81.300813 times the mass of the Moon. Using the Earth-Moon system as a model, a moon with the mass of Earth could orbit a gas giant planet with a mass 81.300813 times the mass of the Earth, less massive than Saturn.
The dwarf planet Pluto has a mass of 8.1967 times its largest moon, Charon. Using the Pluto-Charon system as a model, a moon with the mass of Earth could orbit a gas giant planet with a mass 8.1967 times the mass of the Earth, less massive than Uranus.