Would it be a better world if the law limited what we were allowed to say and what opinions we could legally express?

You have the right to make negative remarks about him - is he not entitled to the same right? What if he honestly believes what he is saying?

We have a great many freedoms in this country, and sometimes it's mighty inconvenient that the other guy has the same ones. I'm not unsympathetic to your situation, but it's hard to put limits on the other guy's rights if we want to keep our own.