Been there for years. They usually argue the liability thing or say they will sign a waiver. I don't like to be a jerk either. The best way that I have ever found to say no is to just tell them that it is for family only.
A few years ago I started telling them that I had lots of expenses like fertilizer, forage, new genetics, shock survey and that I would be glad for them to fish but that it was $250 for a half day (2 people). This year I've gotta go up on that price because too many are handing me cash. It has been nice though, it has help pay for lots of my expenses. It's just business....