If you didn't use his credit card to make the last payment, then you should stop worrying about his accusations. He won't be able to prove that you did.