I believe you can fight the charge as you had no knowledge the card was stolen.

If my friend treats me to pizza, how am I to know he stole the money a week earlier.