My question involves real estate located in the State of: New Jersey

I bought a house about 6 years ago. I had a copy of the survey but couldn't find any stakes so about a year later I had the Surveyor come back and restake the property. One of the spikes was about 18" into the neighbors driveway. I could see that the Belgian blocks had been moved because the blacktop did not go all the way up to them. The neighbors house was a rental. I got the owners info from the town clerk and contacted them. She told me she would have to ask her husband and she would get back to me. After a week or 2 I called her back and she said, "My husband says your surveyor is wrong." They then paved over the driveway and the spike. I had a lot of other things going on with the house and doing remodeling. Now I'm working on the yard and I want to put a fence up so I have to get the Surveyor to come back out mark the corners again. What can I do about the property line? Town ord says I can put the fence right on the property line.