If he sues you, you can attempt to defend against the debt on the basis of coercion, lack of consideration for entering into the note, and that the payments were a gift.