Yeah, the combination of the sudden sound and the ability to feel it happen is what can be frightening. We had some friends over with kids to skate late one night on black ice when the temperature dropped like a stone. It scared the kids right off! They thought they were going to break through. The ice was probably 8" at the time and perfect.
Looking out the window right now, and it is freezing rain on top of snow. Water is leaking up the perimeter of the pond as the weight of the precipitation is pushing down on it. It is making for some poor skating conditions.