The first thing that should be noted is that "straight" is a modern notion. In pre-modern past, relations with members of the same gender were not necessarily seen as unusual as they were for much of the modern period. Homoeroticism may in fact be a better term for these sorts of relations, because they encompassed more than just sexual relations.

Given all the indications in the series, it seems safe to say that Renly Baratheon and Loras Tyrell were lovers. There are few who would argue otherwise, but GRRM has laid matters to rest by stating about them, “Yes, I did intend those characters to be gay” (SSM). It seems safe to say that Prince Oberyn and his paramour, Ellaria Sand, were "bisexual" in modern terms. Ser Lyn Corbray of the Vale was rumored to be "notoriously uninterested in the intimate charms of women" (I: 363), and rumors of pederasty are made explicit in A Feast for Crows.

There is a theory that Ser Brynden Tully, the Blackfish, may be homosexual. This may explain his repeated refusals to wed and Catelyn’s certainty that he never would (I: 659). Others say that this could as easily explained by his having a lost love (some go so far to suggest that he may have been in love with Minisa Tully (II: 363), Lord Hoster’s wife and mother of his children.)

Finally, Daenerys has been argued about following her sexual encounters with one of her handmaidens, Irri (III: 268, 815-816). While these encounters might described as clinical—Irri is providing physical stimulation, no more—Dany’s temptation at the sight of Irri’s breast does suggest at least some sensual feeling in this regards. While she is clearly predominantly heterosexual in orientation, she is just as clearly capable of being aroused by another woman.