Nathan, it's all relative. The pond needs nutrients (fertilizer) for the base of the food chain---it just changes form from microscopic plants/animals to the fish when eaten. Any fish you remove is removing the nutrient. Sooner or later the nutrients have to be replaced, either by you or Mother Nature.