Alternatives to the family could indeed be 'happy' for Butler; for this is how Austin describes the success of speech acts. His research concerns the conditions under which performatives are happy. A connection with 'the family' is soon found. As Butler notes (1993: 224), the (heterosexual) marriage celebration takes an important place among Austin's examples. Now I would expect that Butler sees more opportunity for happy performatives than Austin on this point.
Butler discusses Austin's 'happiness' especially in a treatise in which 'the happy family' seems far away. In Excitable Speech she deals with the manner in which legal condemnation and censorship of insults and threats erode freedom of expression (1997b: 103-126, cf. 1997a: 82). Besides aggressive symbolic utterances such as swastikas and pornography she discusses in detail a debate on homosexuality in the American army. This debate revolves around the question whether one could say publicly that one is gay without this in itself being homosexual behaviour, implying that one would like to engage in homosexual acts. In this context coming out would be an insult or threat. The practical solution found is that one may certainly declare that one is gay as long as one simultaneously promises not to act accordingly. Opposing herself to the rejection of subjects whose desires are made unspeakable by this arrangement, and who are themselves insulted and threatened with the support of the authorities (cf. the analysis of the bio-political overtones of legal exclusion in Agamben 1998), Butler again abides by Derrida's assertion of the unavoidability of deviations at the repetition of the 'insult'. The power of such a performative is not unlimited. She says: 'For instance, I may well utter a speech act, indeed, one that is illocutionary in Austin's sense, when I say, "I condemn you," but if I am not in a position to have my words considered as binding, then I may well have uttered a speech act, but the act is, in Austin's sense, unhappy or infelicitous: you escape unscathed.' (1997b: 16). Butler concludes that Austin makes it particularly clear that performatives fail all the time.
This has consequences for the happy family, and the heterosexual marriage necessary to reproduce it for the benefit of the state. In Austin marriage celebrations fail for various reasons: because someone is already married; in the case of a forbidden degree of family relationship; when the marriage partner is a monkey; by saying 'please don't bother me now, I am in the process of marrying' instead of 'I do'; and perhaps also if the marriage is not consummated. Due to the heterosexual norm it is probable that a marriage with someone of the same sex would be unhappy to Austin as well.
Against my expectation Butler sees no reason here to broaden Austin's conditions for 'happiness'. She considers marriage a repressive state institution, and declares herself against same-sex marriage because striving to gain admission to this institution expands its power, and accentuates the contrast between forms of partnership and kinship that are and are not legitimised by the state (Butler, Laclau and Žižek 2000: 175-177). It is true that marrying enlarges one's own comprehensibility within heterosexual culture, but this happens at the cost of a solidarity with others who fall outside it, such as singles, unmarried mothers and fathers, divorced people, people with relationships other than marriage, with an extra-marital relationship, with multiple relationships, or with relationships considered illegitimate in any way. Briefly, Butler chooses those who did not win 'the prize'.