If you did not have any kind of non-compete, you are free to try to sign his customers. However, he is also free to try to protect his business and keep them. If he tells them you have a non-compete, you tell them that you don't.
I tell you this from experience. Any attempt for you to use the law to stop him from doing what he's doing, is just as likely to end up requiring that you stop doing what you're doing.

