Why on Earth would you even mention this? Also, what the heck does "opposite color" mean?
There's nothing illegal about declining to rent to someone with a significant criminal history. However, since you made skin tone an issue, it is certainly possible that these folks will assume you declined them on that account. There's no way to say this nicely: talking about a a prospective tenant's skin tone (or "race" or national origin, etc.) and pointing out that most of the other tenants have dissimilar skin tone was about most foolish thing you could have done.
You might want to consult with an attorney before you lump more bad decisions on top of the one you already made.

