If you use gravel or rocks in place of the fill dirt in my crayola diagram you'd have nearly total seepage that would mean you�d be standing on water (albeit under some rocks) at the pond edge. Plus the rocks would be more likely to slip down and show the liner�packed fill dirt would be best I think.
I would use clay if you got it just to keep the pond edge as firm and natural as possible...and then sod or hydro-seed on top to get the best erosion control (even with the berm/levee setup you�ll get some dirt washing down in a hard rain and sounds like suspended clay would mess up your plans�I think if you pack and sod/seed well this should not be an issue...if no clay, then just the best packing dirt you can find.
I'm assuming you want the fish bedding in the middle to avoid birds?? and that's why the steep banks?? If so, that's where I'd put the pea gravel and try to create a natural earthen barrier at the waters edge for walking and aesthetics.
