Thanks for the replies everyone. I'll give a few more details. When I bought the property, I had to rebuild the spillway and road on SE part of the dam. The previous owner (a corperation) did nothing with the pond. A beaver family set up camp and blocked the spillway which lead to erotion on road. When I rebuilt the spillway, I also raised the pond about 3 feet. That's where the flooded timber you see in the picture came from.
I'm a little constrained on where I can put the rock due to the steepness, but I'll look around a little more to see where I can place some smaller piles. As for swimming, I plan to put a dock pretty much in the middle of the dam where there is absolutely nothing in the way.