The problem is you don't know it has a rodent issue! You saw some mouse traps in a cabinet but you have no idea why they are there, how long they've been there and even if they've ever been used in that apartment.
Did you see any other signs of mouse damage or just some forgotten traps. What if the previous tenant brought the traps from a previous apartment and never needed them in this apartment so they just left them behind when they moved?
You signed a contract with a non-refundable holding fee, a couple of mouse traps is most likely not going to be sufficient to deem the apartment uninhabitable by the health department. Of course you can always try.