Lou,
Here are my thoughts.
From a water quality point of view, and I'll even toss in aesthetics, I wouldn't want any muddy runoff entering my pond. Certain species of fish are more tolerable of this condition but you don't mention the specie(s) of fish you have.
Looking at it from a siltation point of view, why hasten what time will eventually do to your pond.
In my area, the DEC requires a silt fence to be installed around any areas of disturbance. This past summer, on a house construction site upstream from my pond, a contractor installed the fencing but a couple of hard rains resulted in a lot of muddy runoff entering my pond.
Russ