Size is the main downside. They typically don't get as big as bluegill and shellcrackers.

If pigs could fly bacon would be harder to come by and there would be a lot of damaged trees.