The obvious answer would be yes they can because they did. You should read your CC&Rs to know what powers they have to do that.

However, if your rear neighbor doesn't attach to your fence you...