Did you ask your ex to consent to the passport? It would be a lot easier and less expensive that opening a custody case.