Those "big guys" don't look like "dummys" when running down wind at 25+ knots of board speed in 1 meter chop hanging on to 9 meter sails in 25 to 30 knots of wind.
That's on a 1 meter wide "tanker" formula board. This type of experience seperates the men from the boys.
The "smaller boards" just reach back and fourth in these conditions and can't handle the serious downwind runs (or upwind). Not totally true, but I think you get my point.