If the "harassment" you describe is what others might deem "guilt-tripping", such as "You should be here for our son when he's sick", I doubt that there is a viable cause of action.