What do you imagine a "public building" to be? Private property that is open to the public for the patrons of private businesses? You can be banned from private property. Your remedy would arise if the ban were predicated upon unlawful discrimination (race, gender, religion, etc.) by a typical business.

If you have facts to share, share them.