The house was built by Jacob Rutsen Van Rensselaer,, on a portion of what had been the family's 875,000-acre New York estate near Poughkeepsie. The main residence, a two-story brick building set on an English basement, replaced an earlier structure that was the home of Van Rensselaer's father, General Robert Van Rensselaer. The Colonial Revival style home was designed by the French architect Pierre Pharoux. The interior features a pair of octagonal chambers and features 12-foot ceilings. During the Gilded Age, the house was "Greek Revivaled" with the addition of a white balustrade and a grand porch with pillars, as well as a two-story rear addition built in 1929 and today, the home is roughly 6,000 square feet, and has nineteen rooms. Located behind the house is a former office and two large barns. The mill dates to the 18th century and includes additions completed in the 19th and early 20th centuries. Also on the property are the remains of early waterworks, a store building or former cooperage, a frame dwelling, two additional dwellings, and an Italianate dwelling with four outbuildings dated to about 1884.
Later ownership
In the latter part of the 19th century, Clifford Miller and his family, bought the home and were responsible for the 1920s addition where they held dances, as well as creating the vast gardens, fish ponds, and landscaping the property with specimen trees. In the late 1950s, the family divided the home up into seven apartments so their children could also raise their families there. In the 1960s, it was sold to a real estate company that parceled off much of the surrounding land for development. In 1975, film director and Academy Award winning screenwriter James Ivory bought the home and 12 remaining acres for about $105,000. Ivory lived at the home with his partner, both professional and personal, Ismail Merchant, and with screenwriter Ruth Prawer Jhabvala, who were all principals in Merchant Ivory Productions. They wrote several films in the house and edited some in the apple barn on the property. Since acquiring the home, Ivory has painstakingly restored it to its former glory, and was guided by Jeremiah Rusconi, the Hudson Valley historical expert. Ivory hosted several actors from his films at the home, including Christopher Reeve, Madeleine Potter, Julian Sands, Helena Bonham Carter, Gwyneth Paltrow, Thandie Newton, Wallace Shawn, Vanessa Redgrave, Rachel Kempson, Eartha Kitt, and Madhur Jaffrey. In 1982, the home and surrounding estate was added to the National Register of Historic Places.
