There are many references to the bowels and cells of Casterly Rock, probably stemming from the fact that the land under the castle is tunnelled (I: 104, etc.)
Casterly Rock is allegedly named for the family that ruled it in the Age of Heroes, the Casterlys, who no longer exist (I: 231. TSS: 128)
Casterly Rock sits on rich gold mines (I: 680)
Casterly Rock has never fallen (II: 136)
There are watery caverns deep below Casterly Rock (III: 505)
The Golden Gallery (III: 670)
The Lion's Mouth (III: 670)
The Hall of Heroes, where boys might play, is where Lannisters and their close kin who have died valiant might be interred. From below thunder could be heard, where the sea comes in (III: 670. IV: 497)
Casterly Rock is literally a rock, a castle carved out of a great stone hill, its defenses enhanced with walls and other structures (SSM: 1, 2)