Good groundwork, Joerg. On a purely mundane level I'd agree with Jane and plumb for a nice white limestone; and yes, this would readily allow a variety of water eroded subterranean features. Depending on its makeup, diagenesis etc., limestone is a pretty strong rock also. I know magic can abrogate this, but chalk would make sapping too easy.
A very pure quartzite is another option, as is a more consolidated marble (metamorhosed limestone). A granitic plug, low in minerals such as hornblende and biotite ('da dark bits') might also suffice-- your feldspar. I'm thinking pristine SW English granites here, not the girly pink Aberdeenshire and Finnish stuff most banks seem to made out of these days. ;o)
Granite would be a far stronger rock, all but resistant to sapping, if you want to let mundane matters hold you back. If the plateau of whitewall is lower than local water-capturers, you may develp an artesian well system, pushing the water level up under Whitewall. This would be aided in developing a local aquifer is there was a porous, poorly cemented sandy sediment under whatever what Whitewall was, which could hold considerable amounts of water... especially if you had some non-porous sealing rocks (volcanics, pelites, shales etc.) underneath.
I could ponder further, but it was a while since I did any nonigneous geology! (denigratingly referred to as 'soft-rock crap')
I'll let someone else come up with a mythic, gods' bones reason for all this!
Hope that helps,
Powered by hypermail