The finer stuff will tend to wash down the bank worse, especially on a steeper bank. Stuff too big is hard to walk on.
What I like to do the best is put a thin layer of larger rock, as thin as a single rock thickness. 2-4" stuff I like best. Then a layer of finer stuff over the top. 0-2".
