Inbreeding depression occurs when an isolated population of fish that starts from a very limited number of individuals reproduces through several generations. The general idea is that each individual has some characteristics that aren't beneficial, either in the particular environment in question, or to the species as a whole. If the gene is recessive, it is more likely to pair with another recessive gene, from a cousin fish, yielding an expressed non-beneficial trait. If the gene is dominant, then there is less likihood that the fish with the non-beneficial dominant trait will be outcompeted by a fish lacking that trait.
That being said, stunting of fish populations is more likely an environmental manifestation, as opposed to an indication of a limited or low quality gene pool. The fish you stocked, if indeed possessing poor genetic variability now have your fish to breed with and non-beneficial or detrimental traits will leave the gene pool through simple competition.
You, as the angler, are truly the x-factor in the equation. If you are interested in genetic influences on your fish population keep in mind that you are the top predator and try to assess your impact. I wouldn't worry about inbreeding nearly as much as I would worry about my own influence on harvesting the best fish first.
