I think your on the right track, continue harvesting the smaller bass. If you can catch 20-30 at a time it wouldn't take long in a 4 acre impoundment to make a dent in the population.

Stocking Shad may also be a good idea but it sounds like you have a lot of hungry fish and with so many bass of good size, they could make quick work of them. What kind of cover is around the pond for baitfish now?