I don't think you're in any danger of wiping FA out. FA does provide some cover for small fish and habitat for some small invertebrates. The amount you might want to leave would depend (partially) on how much cover you want to have and how much is available from other sources.

"Live like you'll die tomorrow, but manage your grass like you'll live forever."
-S. M. Stirling
[Linked Image from i.pinimg.com]