What is your #1 goal for this pond?
You mention several things, with both swimming and livestock too. In my opinion, you really only should do/achieve one of those two things.
With your set-up/location of the pond, it looks like choices will be very limited. If you go with livestock area, I think your muck and sediment build-up will be fast. I would dig the most accessible area deeper with a slope towards it,. doesn't matter that you can't reach all of it. The muck will tend to slide towards the deeper end you dig. Then every few years, have it cleaned out in that deeper pit area. Have the area where you can get an excavator and then be able to pull a truck in next to it. That way muck goes from pond to truck with one step. Haul away somewhere else onto your property and repeat as necessary every few years. Initially, you might even have to landscape some spot around the pond to be able to this. but then it will be done for future cleanouts.
The other option is to install a new siphon drain that is set-up to pull water out of the deepest part. Then when the pond fills and flows, it will pull out the muck with the moving water from the bottom. It looks like you have plenty of drop around the pond to achieve a good siphon drain. Not sure if you would be allowed to drop that directly into the road ditch? They might not like the amount of silt that comes with it, however if you had no pond there, that erosion would end up there anyway.
