The correct answers are: I was to have inherited the haunted mansion, but my uncle changed his will at the last minute. I was going to haunt the new owners, but I got distracted by a shiny mirror in the hallway. I would have had to vacate the premises eventually anyway, since the house was scheduled for demolition.

"Was to have inherited" is an advanced structure used to describe a formal arrangement in the past that was not fulfilled.

"Was going to haunt" correctly expresses a past intention that didn't happen.

"Would have had to" correctly uses a past modal to describe a hypothetical future obligation from a past perspective.

"Was on the point to leave" is incorrect; the phrase requires a gerund (was on the point of leaving).

"Should be haunting... yesterday" is incorrect; past unfulfilled obligations require should have been haunting.