I would do as the inspector suggested and if anything happens, deal with it then. It is not cheap to argue adverse possession of even prescriptive easement so unless this is really a big deal to them, I would suggest they will simply acquiesce to the new situation.