The sheetrock gypsum will not harm the water quality. However, I don't know how long it would take to dissolve and doubt that it would completely dissolve. The main drawback that I can see would be the thick paper on the wallboard, because it would take quite some time to break down. Most likely a year or two.

Water dries, rocks crumble, and trees die. The only thing that is eternal is the reputation we leave behind.
- Ancient Viking Proverb