Rutherfurd was one of the social elite nicknamed "The Four Hundred" by Ward McAllister, a number supposedly taken from the capacity of Caroline Webster Schermerhorn Astor's ballroom. Novelist Edith Wharton described Rutherfurd as "the prototype of my first novels". In 1902, shortly after his first marriage, Rutherfurd and his wife Alice contracted prominent New York City architect Whitney Warren to design for them a Tudor revival style mansion known as Rutherfurd House. The approximately 38-room home was located alongside his brother Stuyvesant Rutherfurd's property in Allamuchy Township, New Jersey. Besides the mansion, construction included gardens, a boathouse, a swimming beach, a hydroelectric powerhouse, a 9-hole golf course, and kennels. The farm's 1,000 acres became well known for its Holstein cows and Dorset sheep. After his second wedding, in 1920, Rutherfurd built an estate called Ridgeley Hall in Aiken, South Carolina where he pursued his favorite hobby of breeding fox terriers. In 1907, Rutherfurd won the first Best in Show prize of the Westminster Kennel Club Dog Show with his fox terrierWarren Remedy, who would also win in 1908 and 1909.
Personal life
In 1895, Consuelo Vanderbilt fell in love with Rutherfurd, and Rutherfurd proposed marriage to her. However, Consuelo's mother Alva Vanderbilt forced Consuelo to travel to Europe, and pressured her to marry Charles Spencer-Churchill, 9th Duke of Marlborough, for reasons of title. After Consuelo was persuaded that Alva might suffer a fatal heart attack if Consuelo disobeyed her, Consuelo agreed to forsake Rutherfurd and marry the Duke. Rutherfurd then remained a bachelor until age 40, reportedly having numerous affairs with married socialites, including Ava Astor. In 1926, when Consuelo's annulment was announced, the reason from Rome was that "Consuelo, when 17, and in love with 'an American named Rutherfurd,' had been forced by her mother, Mrs. O. H. P. Belmont, to give him up to marry the Duke." In 1902, Rutherfurd married Alice Morton, the fourth daughter of former US Vice PresidentLevi Parsons Morton and Anna Livingston Reade Street. Her sister Helen, was married to Paul Louis Marie Archambaud, Boson de Talleyrand-Périgord, a son of the Charles Guillaume Frédéric, Boson de Talleyrand-Périgord, Prince de Sagan, and her oldest sister, Edith, was married to William Corcoran Eustis. Alice died fifteen years later in 1917, leaving behind five sons and a daughter:
Lewis Morton Rutherfurd, who died age 16.
Winthrop Rutherfurd, Jr., who married Alice Polk, daughter of Frank Lyon Polk, in 1940.
John Phillip Rutherfurd, who was married to Elizabeth Shevlin and Jacqueline Orr.
Hugo Rutherfurd, who was married to Francesca Villa in 1941.
Alice Rutherfurd, who married Arturo Peralta Ramos, the former husband of Millicent Rogers, in 1943.
Guy Gerard Rutherfurd, who married Georgette Whelan in 1938.
In 1913, upon the advice of Anna Roosevelt Cowles, Lucy Mercer was hired as a social secretary to Eleanor Roosevelt, Cowles's niece. In September 1918, Eleanor discovered the affair Lucy had with Eleanor's husband Franklin. The relationship ended and soon, Mercer then became the governess for Rutherfurd's six children. Rutherfurd soon proposed to Mercer, who was almost three decades younger, and Mercer accepted. She was the daughter of Col. and Mrs. Carroll Mercer of Washington. Only weeks before their wedding, which took place on February 11, 1920, Lewis, Rutherfurd's oldest son, died of pneumonia. Various observers described Winthrop and Lucy as devoted to one another, and their marriage as a happy one. Together, they had a daughter:
Barbara Mercer Rutherfurd, who married Robert Winthrop "Bobby" Knowles, Jr. in 1946.
Rutherfurd died in Aiken on March 19, 1944 after a long period of failing health. Lucy, who was by FDR's side when he died in 1945, died aged 57, on July 31, 1948. In 1966, both Barbara and Franklin D. Roosevelt Jr. denied the relationship between her mother and his father.