I had the exact same situation the year before last...had a 6.5 year old buck, GIANT body, but a fairly small 10 point rack. Bad genetics. Field dressed right at 275. Huge body, small rack.
And I realized last year there are literally dozens of his progeny running around the area. I have a whole slew of good bodied bucks with weak genetics on the rack.
I'm not totally against that...I'm in this for the freezer first and the wall hanger second. I assume that the gene pool will also increase the size of the does as well, which are my main source of red meat throughout the year.