3f farms,
Up here in SD we have a couple ponds in the 25 acre range, and the black dirt ranges from 1-3 ft in depth across the bottom. This has caused no problems for me in terms of water being muddy. However if we had carp or other bottom rooting fish in the pond, that might be another issue.
I think a more important issue for you might be the water depth vs how much submergent weed growth you will have. In our ponds it takes a depth of 10-12 feet to keep submerged weeds from growing to the top of the water column. While some weed growth is useful for escape cover for young fish, too much can interfere with your enjoyment of the pond, and could contribute to winterkill (if you get ice cover).
Up here they recommend keeping weed to less than 25% of the surface area of the pond. We've got more than that in some of our ponds, and I'm now looking at getting in a dragline and excavator to deepen certain areas. It would have been much easier to move this dirt before the pond filled.
Thus I'd suggest that while you have the opportunity to move dry dirt, do it, if you can afford it. If you can get someone with a dump truck and excavator, you could move a lot of dirt in a hurry. You could also excavate the dirt in patterns to create some trenches, ridges, humps, holes, etc.
One think to keep an eye out for, however, is any porous materials that may underlie the black dirt. You don't want to dig into some type of material that could be connected to a passageway for your water to flow around or under your dam.
Bill