My question involves a child custody case from the State of: Missouri/Colorado

I have a question and was wondering if anyone had some advice.
I have sole physical and legal custody of my daughters. My ex-husband has supervised visitation only at my residence. He recently chose to move to CA to be with a new girlfriend he met on the internet and has only known for a month. He was not utilizing his visitation even when he was here in Missouri. He does not pay child support which has been ordered. We are moving to Colorado to stay with family so I can get on my feet.
My question is--how can I protect my children from his bouncing in and out of their lives? I am happy he is gone due to domestic violence and partying issues. But I am afraid when he becomes bored with his "new" family he will come looking for them. I have read quite a lot about termination of parental rights here. I am a single mom. What chances do I have of this?
I know he has rights. But don't they? I am worried about the effect he will have has they grow older.
He has told me recently that he and his girlfriend will file for visitation in CA. I would think he wouldn't get it but I am wondering if anyone here has had any experience with this type of situation. Neither of them are responsible enough to care for the girls and I know this is just a threat.
Just to make it clear, I don't hate him, don't want to punish him, just want my girls to have stability. He doesn't care. Looking for some sound advice.