I suspect that the only real limit is collapse under its own gravity. Of course, it would get more and more fragile as it reached this limit. But, it's floating in empty space. It might help if it was spinning about its long axis. And it's not just length but length to width ratio. I would guess that a few miles across and a hundred miles long would be possible. But very fragile. (Answer partial based on looking at the size of existing asteroids, eg Ceres is approx 1000Km, but gravitationally collapsed).
Addendum - response to comments.
Rock has a tensile strength of 10 MPa and is brittle. (No, I am not going to attempt any calculation). A bending rod gets compressed on one side and stretched on the other. At some curvature, the stress on the outer edge exceeds the tensile strength, and a crack propagates suddenly across the rod. For a natural space object, the surface would already be riddled with bumps and dents, which would greatly weaken the rock.
As the rock gets longer, think of it as two halves with a center of mass at the 1/4 and 3/4 points. The mass goes up with the length, and the gravity force down with the square length, meaning that the gravity force is getting weaker. But, now think of the rock as curving slightly, with a uniform curvature that is less than the snapping point of the rock. As the rock gets longer, the lever arm of the force gets greater, and the stress on the outer edge larger in proportion to the force.
When the rock is fairly short, this is a 2nd order effect, we can dismiss. But as the rock gets long enough to bend noticeably, actually the distance between the two centers no longer goes up with the length of the rock, and the lever arm eventually does go up with the length of the rock.
Precisely what the balance is and where the breaking point is might be difficult to determine by an exact and justified calculation.