If his rights have truly been completely, legally terminated - as opposed to him relinquishing custody - then you do not need his permission, because he is no longer their father.

Why doesn't your husband simply adopt your children?