I also have a similar situation. We moved in about five years ago and the pond was overcrowded with small bass, lots of big bluegill but virtually no small bluegill and about 50 small (12") CC. We also caught a few very large crappie. I assumed at the time that they were all the same sex because we only caught two or three a year with pretty heavy fishing and they were all very large. I began removing all small bass and feeding the cats and bluegill. About three years later we had lots of bluegill in all sizes, the bass were starting get get larger and we had CC up to 10 pounds.
Then the milfoil showed up. Almost completely covered the pond. The pond is very fertile because the runoff comes from a horse pasture. I had to use chemicals to control the milfoil. It is now mostly under control. One day if Feb. I was fishing when the water temps up here were probably still in the low 40s and I started catching crappie. I was surprised because I had never really caught many crappie. Anytime I fished next to woody structure I was catching 6 - 9" crappie. Still lots of bluegills including small ones. In the last two months we have taken out between 100 - 200 crappie. I have caught a couple big ones that were thrown back. The bass are still looking good with some pushing two pounds. The crappie population seems to be under control since we are not catching as nearly as many as we were. This may also be due to the fact that they are probably done spawning. The ones that we were catching were either very dark (males in spawning color) or had eggs.
As long as they don't seem to get stunted I will try to keep all the small to medium sized crappie we catch and throw back the really big ones to grow. I was going to try stocking tilapia this year, but apparently they are not on the Illinois approved stocking list. So I am going to try stocking some Golden Shiners and grow some out in horse troughs to use for stocking and bait.