I'd say it definitely depends on the trail. The only thing I can think of that has not already been mentioned is following in a sea of dust. That can spread out a group quite a bit as well. And is progressively less fun the further back in the pack you may be.
'Driving conditions' aside, it becomes a matter of handling the number of people in and out of campsites / restaurants / bathroom breaks / photo-opts / etc. So with that in mind, and vehicles seating between 2 and 4 persons, I would think 5 to 10 vehicles would be acceptable, if you could get that many to commit to going on a regular occasion.
And personally, bigger groups are more comfortable for new people. Better chance of personalities lining up
