My question involves a child custody case from the State of: California

I have physical Custody of the children. Legal custody is joint. All children related issues were resolved as part of our divorce settlement, which was accepted in full by the judge as part of the final judgment.
A few important points:
- the agreement allows me to relocate with the children out of State, without consent from the NCP, as long as I give written notice
- however as part of the joint legal custody, NCP has a right to participate to decisions relating to important events in the children life and I assume that relocating out of State would be considered an important event...
- child support was quantified based on the children spending 90% of time with me
- NCP never paid any child support and arreages are now quite high (about 5000 dollars)
- NCP never respected the agreement and never kept the kids the 10% of time he should have
- NCP sees the kids occasionally - a couple of hours per week on average
- visitation rights of the NCP simply states that he should have access to visit the children with 24 hours written notice as long as reasonable - NCP would not agree to any more defined schedule.
- NCP has left his job and refused to work for the last several months

Now, my company is relocating my job out of State and I am required to relocate with it. I am providing for the children financially (and not only) in full, on my own.

I feel that the above described situation should allow me to relocate easily without having to go to court of filing an OSC.
However I fear that the NCP may try to go to court to stop me.

Any advice?