I think just slipping in a few HSB, 2 in spring, 2 in the fall, will prevent an ecosystem crash. Ladder stocking if you will.
What I don't think you want is putting in too many HSB, and they empty the goldfish forage out in a very short period of time, then starve. You want a balance where the fish are reproducing just a little bit slower than consumption while you also try to get a new forage base established, like bluegills or some other panfish.
Too many predators, and you will have trouble keeping them happy once the forage base is gone.
