My question involves adoption law for the State of: Florida

My ex and I have three children together. We married in 99 and was divorced in 2007. My current husband and I moved due to him being military. My ex continued to see and visit our children. We would meet half way and talk alot on the phone. He started dating a woman who hates kids. He drove 1100 miles to move her to him. My ex picked our kids up on July 12th, 2009 and took them for a three week visit. He was supposed to have them longer but "they" had plans. They being him and his girlfriend. I didnt fight about it, I just went with it. On July 13th, not even 12 hours after he picked them up, I recieved a text message that said I am bringing them home. Well he slept on it and desided to keep them. He traveled to texas with them and within a 24 hour period they had two trips to the ER room. Once because my daughters ear got hit and was hurting and the second time, my son had called me because my four year old daughter was choking on a sun flower seed and I had to beg him to take her....after he tried to ram his finger down her throat saying there was nothing there. Our divorce papers state that he is to pay dental and insurance on them. I just got notice that he dropped the dental so far and now I owe a ton of money and they wont finish the work on my son till they are paid. Anyway, I got away from what I was saying, I apologize. I met him on July 28th and picked them up. We spoke the next day and he has not called or anything since that day. I had my four year in the emergency room with a fever of 105 and he didnt call or anything. I texted, emailed, called his family...everything...and got no response...He does pay child support but does not call or text no regular mail..nothing..how long do I need to wait before I ask a judge to terminate his parental rights or because he is paying child support does that mean he doesnt have to be there emotionally for our children? I know its only been a short time, but his girlfriend does not want the kids anywhere near them, so I dont think it will change anytime soon!

Thank you