One of my favorite ways to hunt is stalking with only my handgun.
Only do it if it's raining as I don't like sitting in the rain and it works great as your rifle doesn't get wet and the handgun can stay under cover unless you have a shot.
You really can move silently in the rain and without a long barrel to carry, you can keep a low profile. It is a very exciting way to hunt.
All that being said, I've never killed a deer this way. Had some questionable shots I didn't take...it seemed more exciting to see how close I could get.
Maybe this year. I use a 1911 that I'm confident with for an appropriate shot.