I don't know if you've given this any thought or not, but has the Westeros realm ever invaded the mainland to the east?

There have been a few attempts to claim some of the Stepstones, a chain of large islands in the narrow sea east of Dorne and Storm's End.

And after the death of the Targaryen dragons, did the lords ever rebel against their king?

Yes, there have been rebellions, most notably by the Blackfyre pretenders.