With all that we might have been better off just to have stayed home.
It should be 始めから, not 初めから. It’s a common mistake among native speakers. 始め means “beginning” or “starting point” whereas 初め means “early” or “the first”. If you say 初めから家にいたほうが良かった, that means you repeated the same situation again and again, and you are now describing the first time you did it.