Я женат, и у меня двое сыновей.

English Translation

I am married and I have two sons.

I thought 2-4 were genitive singular

Ah, but двое is a collective numeral. And according to its Wiktionary page:

Дво́е (Dvóje) is used with nouns that only have a plural form (pluralia tantum). It can also be used to denote a number of people. The nominative and accusative cases of a collective numeral govern the genitive plural of the noun, pronoun, or adjective. The other cases agree with that of the noun, pronoun, or adjective.

I asked a native Russian speaker whether there’s a difference between два сына and двое сыновей. He said the two forms are equivalent.