Mrs. Winchester was deeply upset by the deaths of her husband and daughter and seems to have consulted a spiritualistic medium. Reportedly, the medium explained that the spirits of all those who had been killed by the rifles her family had manufactured, had sought their revenge by taking the lives of her loved ones. Further, these spirits had placed a curse on her and would haunt her forever. But the medium also stated that she could escape the curse by moving west, buying a house, and continually building on it...as the spirits directed. In this way, she could escape them and, perhaps find the key to eternal life.

Whether Mrs. Winchester believed the medium or not is unclear, but she did move to what is now San Jose, California in 1884 and purchased an eight-room farmhouse from Dr. Robert Caldwell. Dr. Caldwell came to San Jose from Kentucky by wagon train. He built the farmhouse in San Jose and lived there with his family (wife and 9 children) until he sold it to Mrs. Winchester. The family remained in San Jose from 1848 until at least 1932, when Dr. Caldwell's daughter Caroline died.

Mrs. Winchester immediately began her never-ceasing building project. With a great deal of money and very few responsibilities, she satisfied her every whim and notion by keeping a staff of 18-20 domestic servants, 10-22 carpenters and 12-18 gardeners and field hands constantly busy. She had no master plan for a house and according to her carpenters, built whenever, wherever, and how-soever she pleased, always directed by the spirits. As a result of the constant building, tearing down and remodeling, the mansion's form shouldered its way ever higher into the skyline as its great bulk crept over the surrounding acres, engulfing several outlying structures over the southeast section of her 161 acre estate. She built steadily, 24 hours a day for 38 years, until her death in 1922.

During the 38 years Mrs. Winchester worked on her mansion, local people would pass by the estate and would wonder at the strange and mysterious construction. Many would try to explain it to others. From one telling to the next, many strange stories would arise.

Frequently, these stories would be recounted to Mrs. Winchester and, according to her niece, Mrs. Marion Marriott, would upset her very greatly. Many of these stories have been retold so many times, they have become part of the legend of the house.

Some feel Mrs. Winchester was a very sane, although eccentric, person. If she had actually believed that she would have eternal life if she kept building, why did she leave a will? Those who worked for her usually stayed for 15 or 20 years. One carpenter stayed and worked for 36 years.