I am NOT an aeration expert, but that answer seems a little "too easy".
If you shut down in the spring with actively growing photosynthesizing organisms, and cool water that can hold lots of dissolved oxygen, then you are probably safe for a fair amount of time.
If you shut down in August, during a heat dome, with no wind, and your highest water temperatures of the year, then your "safe time" will definitely be much shorter!
If an algal bloom dies off at the same time and starts consuming oxygen, then you could definitely experience a fish kill.
Do you live at your pond? If so, then you observing the daily conditions and making the appropriate adjustments would probably give you a better safety margin.
P.S. Is it relatively easy for you to move your aeration stations? If so, then you might experiment and see if you can still get oxygen into your water yet not create quite as much mechanical agitation of your particulates.
Hopefully, some of the aeration experts will drop into your thread.
