In order to get a big earthquake, you must move a large fault plane. On Earth, the biggest earthquakes happen in Subduction Zones where you get shallow-angle thrust faults; these can have moving surfaces measuring hundreds of kilometers to a side. The energy release is proportional to the area moving and the distance moved - but as a bigger fault area can take more strain before moving, the distance moved is proportional to the fault area. Overall, then the earthquake energy rises as the cube of the fault length. (See eqn 15)
SO, what does this mean for your planet? Assuming similar composition to Earth, the factor is actually going to be the presence of subduction zones, and/or the geothermal gradient. The reason being that on Earth, subduction zones give anomalously large earthquakes, dwarfing any others by at least an order of magnitude. If your planet has subduction, then that's where the biggest ones will be.
Outside of that, the key is the geothermal gradient. Earthquakes can only happen in brittle rocks; and at a certain temperature, rocks become sufficiently ductile that faults move by a aseismic slip. On earth, this limits the size of earthquakes by limiting their vertical rupture length - so the San Andreas fault cannot generate earthquakes greater than magnitude 8, for instance.
Your fictional planet is smaller than Earth, and therefore it might have lower internal heat generation and a lower thermal gradient. This in turn would allow for larger fault depths and therefore bigger, but less frequent Earthquakes. Perhaps up to magnitude 9 on a strike-slip fault like San Andreas, or magnitude 7 or so in an area of extension like the basin and range province. The increased density (how?) would probably drive tectonics faster.
So, overall, assuming earth like composition (in the surface few hundred km at least), les frequent Earthquakes of a greater magnitude than Earth. Add 1-2 richter numbers to the ones you'd get in a similar tectonic setting.