The best thing to do is get a #330 conibear trap. Put it in the opening of their den and leave any beavers you catch in it for a day or two. The others will have to swim around the dead one. This tends to scare them off.
Some states require 330s to be used underwater only.
You�ll want to use two long stakes to hold the trap in the den hole