If you say nothing one way or the other then you won't be seen as either supporting him or defending him on the issue.
If you want to play odds, I suggest you go to Vegas. You have no idea whether the claims of this particular woman are true. Even though most women don't lie about this stuff, some do and you don't don't know which one this woman is.
Because falsely stating that someone is a pedophile or a sexual harasser is defamatory. Freedom of speech does not allow you to harm the reputation of others without consequences. And the consequence is that you may find yourself sued for defamation.