If the non-compete doesn't mention a period of time in which you may not "compete" and a geographic limitation (say, "within a 15-mile radius"), then it won't be enforceable. What you quote here isn't much of a non-compete. Even if it does have a time limit and geographic limitation, if you believe you have a good argument that this other company isn't a competitor because your company could never handle clients the same size, I say that's a reason to leave.
"Do I have to tell them where I'm going?"
You're asking US? We cannot know from here whether you obligated yourself to tell them where you're going; of course, we can't know what a prospective employer will do in terms of checking references.![]()

