I would guess that if the bottom of your trench is above the full-pool level, then the risk of a problem is low. I would still tamp down the backfill in layers so it does not provide a weak spot in the dam.