I've not heard that fresh clay and harm the fish, but would be interested to hear from the more experienced on the topic. With that said, I would consider letting the freshly renovated pond fill naturally for the most part, then pump the temporary holding pond water back into the main pond. This would allow for some aging and settling to occur of the new hole and get the holding pond down enough to seine out the fish for transfer. Consider putting FHM's in the renovated pond as soon as it has a few feet of water in it along with some floating spawning structure. Given a few months before adding the fish back could produce many minnows for forage and seed the pond biologically. All this would depend on how well the fish are doing in the holding pond.

Fish on!,
Noel