Do you hold joint accounts? Bank accounts, credit cards, utilities? Does he come home to you regularly? (I say "home to you" since you both started out in the same state and he was forced to relocate for his job.)

Document, document, document, and have a chat with an Immigration attorney.