I used to be one of those dirtbike riders(long time ago)I rode all through the neighbors pastures. Never broke or tore up anything(not counting me or my bike). Heck, the owner was never around to meet or ask. All we knew was, he was some rich bigshot from Dallas, that only came down periodically. But we were always taught not to tear up other people's property. We were also smart enough to know that if we did start tearing things up (fences, barns, etc) We WOULD get to meet him(and the local officials). So ride was all we did. But nowadays these kids are not taught the same (from my perspective). Good luck to you however you handle this, just make sure you've thought your action through.
