Using Bill Cody's term -- "it all depends."
I downsized, and downsized, until I'm now down to a 23 HP 4WD. As of next week, I'll have had it for five years. I've got 25 acres.
It is a real heavy little tractor. It has a front-end loader, and I've got the factory frame-mounted backhoe for it. The rear tires are loaded. It weighs in at about 5000 lbs. when the backhoe and front-end loader are on it.
All of my implements are 5-6 foot. My box scraper and brushhog are 5-foot. My rake and rear blade are 6-foot. It has all the traction I need for pulling a box scraper full of gravel up a very steep driveway. Best of all, it is extremely maneuverable. Things like a post hole digger, or a finish mower, require very little PTO power.
Except when I'm using the backhoe or brushhog, is uses less than a gallon of diesel per hour.
Now, I'm probably more fortunate than most. I have a good friend with a 65 hp 4WD tractor, a big skid steer, an excavator, a dump truck, and trailers to haul it all. Frequently, they are stored at my place. However, even without them, I'm still able to do 90% of my projectgs with my little-bitty tractor.
I'd only get a larger tractor if I had to mow lots of acreage several times a year and had to bale it. If I just had to mow it, I'd just get a 54-inch conventional or zero-turn mower.