Horsley married Elvira Walter in 1846 with whom he had three sons: Edward, Frank, and Harry. Elvira died of consumption in 1852 followed by the deaths of Edward and Harry in 1854 and Frank in 1857 due to scarlet fever. Horsley remarried to Rosamund Haden who came from a family of distinguished surgeons—her father Charles Haden had a practice in Sloan Street and her brother Francis Seymour Haden was a surgeon and etcher who founded the Royal Society of Painters, Etchers and Engravers in 1880. Rosamund gave birth to Walter, Hugh, Victor, Emma, Fanny, Gerald and Rosamund. Gerald grew up to be an architect, Walter an artist also studying at the RA Schools, and Victor a surgeon. Sir Victor Horsley, became famous as a surgeon and neuropathologist, and a prominent supporter of the cause of experimental research. Horsley and Rosamund had four more children: Emma, Fanny, Gerald and Rosamund, losing Hugh and Emma to scarlet fever. After his wedding to Rosamund in 1854 Horsley and his new wife toured the Midlands for five months to establish contacts with wealthy industrialists for portrait commissions. Horsley moved into 'Willesley', his house in Cranbrook in 1861, joining the Cranbrook Colony; whilst maintaining a home in London. The architect Richard Shaw adds "...tall chimneys and cosy 'inglenooks.'" in the Jacobean style to 'Willesley'.
Career
Horsley's paintings were largely of historical subjects set in the seventeenth and eighteenth centuries, influenced by the Dutch mastersPieter de Hooch and Vermeer. Examples are "Malvolio", "L'Allegro and il Penseroso", "Le Jour des Morts" and "A Scene from Don Quixote". As a young artist Horsley was patronised buy the collector John Sheepshanks, who bought two of Horsley's paintings: ', and '; both of which are now part of the V&A collection. In 1843 his cartoon of "St Augustine Preaching" won a prize in the competition to provide interior decorations for Palace of Westminster. This led to his being selected in 1844 he was as one of the six painters commissioned to execute frescoes there. He painted "" in the House of Lords, "Henry V assuming the Crown" and "". In 1864 he became a Royal Academician. Horsley had much to do with organizing the winter exhibitions of "Old Masters" at Burlington House after 1870. Horsley was rector and treasurer of the Royal Academy from 1875 to 1890 and 1882 to 1897 respectively. He earned the nickname 'Clothes-Horsley' for his opposition to the use of nude life models. When, during the 1880s, the example of the French Salon began to affect the Academy exhibitors, and paintings of the nude became the fashion, he protested against the innovation, and his attitude caused Punch to give him the sobriquet of "Mr J. C Horsley". He resigned from the academy in 1897, and became a "retired Academician". Horsley designed the first ever Christmas card, commissioned by Henry Cole. It caused some controversy because it depicted a small child drinking wine. He also designed the Horsley envelope, a pre-paid envelope that was the precursor to the postage stamp. In 1856 Horsley was photographed at "The Photographed Institute" by Robert Howlett, as part of a series of portraits of "fine artists". The picture was among a group exhibited at the Art Treasures Exhibition in Manchester in 1857. Horsley was a member of the London-based Etching Club contributing illustrations to editions of "The Deserted Village".
