Agree with esshup on the mini-excavator.

They are quite easy to operate for a newbie.

Observations:

1.) Most mini-excavators have two "control modes". For amateurs, it is usually more intuitive to have the rental company set it up to operate in "backhoe" mode.

2.) They are safe to operate for newbies if you do NOT drive or operate on slopes. (Also only safe where there is no chance of hitting underground or overhead utilities.)

3.) I suspect you will need to operate from a flat position on top of the dam. You will need to measure how far to reach to dig out most or all of the beaver tunnels. In my experience more HP and reach is almost always better on renting a mini-excavator. The rental rate only goes up slightly as you get more HP. The work accomplished per hour versus the rental cost per hour has always been better for me with the larger machine.

My limitation is generally the hauling weight. I only have a beefed-up 1/2-ton pick up. It is flat from the rental business to our farm, so I usually get the excavator that is right at my towing weight limit.

Get the online specs for the largest machine you can haul. See if the "reach" in the specs will fit your project needs. If not, then you may need a buddy with a 1-ton truck so you can get a bigger machine (for example).

4.) Packed clay on a dam face will probably be very hard digging. In that case you want a smaller bucket to maximize your break-out force at the cutting edge. For example, if the beaver tunnels are 10" wide, I would ask the rental company to fit their 12" bucket. You will also need a bucket with teeth. Tell the rental guys your project when you reserve the machine, and they should provide you the optimal set-up for when you pick it up.

I hope that gives you some more ideas for safely and efficiently fixing your problem.