At the present time, no, you do not since you have not suffered any actual damages. It's barely possible that in future you might IF at some time this comes back as you fear, but even then you'll have an extremely tough row to hoe because if he sincerely believes that you are racist, he has the right to state that opinion even if he is wrong.