The marriage is valid, even if it was not valid or was questionable at the time the marriage was enter into because of the duration of the marriage its validity has been accepted. A single incident, or even a serious past incident, will not be enough to prove that you are an unfit mother. The courts (in my experience) usually frown upon one party attacking the other when there is clear evidence to the contrary. Are you currently providing a reasonable home for the children? If the answer is yes, then you are okay. If your ex brings up something in an attempt to embarass you he will have to answer some tough questions...
This happened x years ago yet you remainded married and through your actions choose to forgive or overlook the incident, so why do you bring it up now?
If the mother is unfit then why have you allowed the conditions to be such for an extended amount of time?

