heronblu, you are to be commended for being a good neighbor. A simple phone call to ask, is reasonable. Only then, I would let folks fish. And I would tell them, if I find one piece of trash, I will assume you did it, and you won't be allowed to fish again. This way, they will pick up others trash.
Also, you probably should tell them, you want their help with a few chores around the farm, in a trade for fishing. Like fence work, etc.
My guess is, your neighbors want something for nothing. They want to fish, but won't want to help you around the farm. I would tell them, every time they ask, that you need this or that done, if they are coming fishing anyway. They won't come around, I'll bet. Again, something for nothing. The one or two good neighbors, who will agree, well you'll find out who they are, and they can start really keeping an eye on things for you.
You need to become a good actor. Yell on the phone "HEY YES, COME ON TODAY, AS A MATTER OF FACT, I NEED HELP WITH WORKING THE COWS, PUTTING HORSE SHOES ON, OR FENCING TODAY, OR SATURDAY, CAN YOU HELP ME OUT, WHEN YOU COME TO FISH? IT SHOULDN'T TAKE BUT A COUPLE OF HOURS. I'll bring a sandwich for lunch and gatorade".
Put on your acting hat. They will disappear. The poachers will become evident, as well as the good neighbors.
