I would plant the annual rye to hold the soil in place. You can also look at planting any of many wild flowers, sedges and grasses that like wetter areas to add interest to your pond. There are many ideas in some of the other threads on here.