I think you can have swimming and fishing, that is what we did with our third pond.
For swimming I wouldn't ad BG at all, YP may be a better choice, catfish dirty the water also and are super predators. Stone or small rip/rap around edge will help keep water cleaner for swimming.
Starting with a good forage base like FHM, GSH, YP, RES first to get them spawning will help. I wasn't wanting to feed when we first dug our swimming pond either but it is really enjoyable. We hand feed off the dock most mornings and evenings during the warm months and it is a hoot to see them come up.
| 
