I think your approach to the neighbors has been good. As always, time will tell as to who the truly good neighbors are as far as really caring about your land.
If there's a opening day to Morel season, I would be at your property early that morning and walk the property and politely "meet" each person you see. As ewest has said, it's always good to have friends with you (armed or not is your choice; I prefer armed). Again as ewest says, it would be a good idea to get a photo from a distance. Then I'd tell them that the property is off-limits now and kindly ask them to step off. I'd do the same thing for many random days throughout this 'shroom season.
Returning to the neighbor thing, if you end up finding one of your neighbors is not who you thought they were, you can change or eliminate any agreement you have with them. I believe that direct & clear communication is best for that; too often, I feel that neighbor problems are exacerbated because people don't actually make enough effort to talk things out.
As far as deer season, that's a really tough call whether you want to be out walking your property (as a newcomer) in-season until you know whats really been going on.
The best time to do so would probably be in the weeks leading up to deer season while hunters are scouting out locations and patterns.
