SethM, you can make an impact on Crappie populations by fishing that will make a difference in your pond.
Right around this time of year, maybe even a week or two back, they should be schooled up where you can catch several dozen in an hour with a little jig. Take out every one you catch, or use a slot limit.
Another potential option, with side effects, would be to stock a small number of tiger muskies.