Two ducks good . . . Twenty ducks bad.
I don't know what you could do at this point, other then wack a bunch of them and don't fertilize for a while.
You should probably have some emergency aeration on standby.
If you want to have ducks in the future, I would definitely install a aeration system, preferably a subsurface air-diffuser system (not a fountain). Check the resources portion of this site to find a dealer.
