Here's another rule of thumb I've adopted:
If an animal spends more time in a location than I do, I consider that location the animal's home. Meaning, I'm just a guest there and I should not be overly intrusive.
On the other hand, if you are in a specific location more than a particular animal, it is likely you won't have to perform a removal. That animal, if smart, will be gone before you can take drastic measures.
