they most likely will be able to enforce it. Just to make sure, contact a attorney to review the clause and advise.

Also note that there is no such thing as 'permanent employment' so the notation you made is irrelevant. Most employment is deemed at-will which means they can let you go for no reason or any reason at all; barring those protected by law (age, race, religion). If you signed the agreement, you're bound by what you've signed.