IMO the ability to kill a hog is the thing that gonna limit your ability to go with a smaller round. You can kill a hog with a .22lr, but to efficiently kill a hog you need a little more oomph...
My dad has a .204 Ruger and I am very impressed with the ballistics and accuracy of the round. Even though it is a small caliber, a well placed shot would easily kill a hog. A sub par shot may not be so good. 6.5 -7.5 pounds is a fairly lightweight gun and in a larger found you're gonna get some kick from the recoil. The bigger rounds will be a bit over kill on prairie dogs.
Going on your requirements, I'd go with the 7mm-08 as a do it all round. But I sure do like the Ruger .204.
