They live in separate states, and are going to continue to do that after the marriage, for at least until the 2 years it takes to get her green card.
Because they don't think that the USCIS is going to want confirmation that they are living together legitimately as man and wife? They ask for things like leases and utility bills and bank statements and all manner of other proof that they're living together as man and wife.

They even ask if the marriage has been consummated.

He wants him and I to continue seeing each other.
You told him to go to hell, right? You can't possibly think he actually cares about you, to expect you to go along with this fraud?
My question is, will they find out about me? And if so, what happens?
USCIS doesn't care if he's got "one on the side" - having a girlfriend doesn't torpedo his "wife's" marriage based green card. Not living legitimately as man and wife does.

So, WHEN USCIS finds out (and they WILL), she will be deported and likely banned, and he will be prosecuted for immigration fraud, which carries huge fines and prison time - $250K and 5 years.
I kind of hope they do find out, as i do not want him to do this and have been trying to talk him out of it.
Speed it along. Turn them in and DTMFA. You can do better.