Raking the FA out will remove it's nutrients (deterring future plant growth) and prevent it from consuming pond O2 as it decays. It's a plus (especially in eutrophic waters), but your call as to whether it's worth the effort.

"Live like you'll die tomorrow, but manage your grass like you'll live forever."
-S. M. Stirling
[Linked Image from i.pinimg.com]