Fish love all sorts of worms, there shouldn't be any issues feeding your fish worms. The worms are probably a better protein source than the fish food you are feeding.