You really need to know what you signed. A confidentiality agreement is completely, utterly and totally different - like really really different (got that?) - from a non-compete agreement. If your aim is to start/work for a competitive business, then a typical confidentiality agreement won't prevent you from doing that. So if that's what you signed, you have no issues.
But if you signed a non-compete agreement? Maybe some problems.
So you really need to decide what you signed before you start possibly wasting money on attorney fees.

