
Quoting
Mr. Knowitall
Intent can be inferred from quantity alone. If you know it's there, and they probably have or will try to have informants make buys during which you may be (or may have been) present, there is a substantial chance that you'll be charged as an accomplice, accessory, or partner in crime.
Your choice right now seems to be between having a drug-dealing roommate because you like having ready access to the drugs, or minimizing your chance of being charged with serious criminal offenses.