The earth material in our area varies greatly over very small areas.
I'm guessing you have mostly clay, but you could easily have a vein of gravel or sand coursing through it. Especially, since you are feeding it with a spring. You could have even hit a spring when you were initially hand digging.
Are you digging the remainder out by hand? If so, that could be somewhat of a problem. But, if you have a backhoe or small excavator, the clay that is pulled out could be used to help seal any sand or gravel veins.
On my land, I've got areas where a hole can be dug in totally impervious clay. It doesn't have to be compacted. Water only gets out by evaporation. But, less than a hundred feet away, I can dig another hole, where I hit shale and gravel that absolutely won't hold water in even the heaviest rains. My current burn pit is about 100 feet from my larger pond. The pond doesn't leak. The burn pit, which is about 8-feet deep, can get as much as 4-5 feet of water in it during a big storm. When it quits raining, it will have no water in it four hours later.
Take earth samples everyplace you plan to dig. Compress them in your fist to squeeze out as much water as your can. If the sample holds together, the soil will probably hold water. If it crumbles or falls apart, it will have to be dug out, or covered with impervious clay.
Good luck,
Ken