How about getting him to confirm in writing (even if by email) that he has abandoned the premises and has no plan to move back in and, if he won't return the keys, then getting your landlord's permission to change the locks? Has he indicated instead that he intends to try to move back in?

