On 4 March 1767, the Duke of Cumberland allegedly married Olive Wilmot, a commoner, in a secret ceremony. There reportedly was one child, Olivia Wilmot, from this relationship, though the duke's paternity was never proven, and Olivia Wilmot was accused of forging the evidence. A landscape painter and novelist, Olivia Wilmot married John Thomas Serres and later, controversially, assumed the title of "Princess Olivia of Cumberland". Cumberland's mistresses included Ann Elliot who had been an actress before another had taken her off the stage. Cumberland set her up in a house in Greek Street in Soho where she died after an illness in 1769. Cumberland arranged for her burial and memorial and gave a large sum to her estate. In 1769, the Duke of Cumberland was sued by Lord Grosvenor for "criminal conversation" after the Duke and Lady Grosvenor were discovered in flagrante delicto. Lord Grosvenor was awarded damages of £10,000, which together with costs amounted to an award of £13,000.
On 2 October 1771 the Duke married Anne Horton, daughter of Irish peer and British MP Simon Luttrell, and the widow of Christopher Horton of Catton Hall. The marriage caused a rift with the King, who considered it a mismatch, and was the catalyst for the Royal Marriages Act 1772, which forbade any descendant of George II to marry without the monarch's permission. There were no children from this marriage. The marriage between Anne Horton and the Duke of Cumberland was described as a "conquest at Brighthelmstone" by Mrs. Horton, "who", Horace Walpole says, "had for many months been dallying with his passion, till she had fixed him to more serious views than he had intended." Anne was however generally thought one of the great beauties of the age, and Thomas Gainsborough painted her several times.
Later life
In 1775, the Duke established the Cumberland Fleet, which would later become the Royal Thames Yacht Club. He was promoted vice-admiral of the White in 1776, admiral of the Blue in 1778, and admiral of the White in 1782, though he was forbidden from assuming any command. The Duke was also instrumental in the development of Brighton as a popular resort. He had first visited in 1771, and in 1783, the Prince of Wales visited his uncle there. The Duke of Cumberland died in London on 18 September 1790. His widow died in 1808.
Titles, styles, honours and arms
Titles
7 November 1745 – 22 October 1766: His Royal Highness Prince Henry
22 October 1766 – 18 September 1790: His Royal Highness The Duke of Cumberland and Strathearn
The prince's full style, as recited by Garter King of Arms at his funeral, was the "Most High, Most Mighty and Illustrious Prince Henry Frederick, Duke of Cumberland and Strathearn, Earl of Dublin, Knight of the Most Noble Order of the Garter".
Arms
Henry was granted use of the arms of the kingdom, differenced by a label argent of five points, the centre bearing a cross gules, the other points each bearing a fleur-de-lys azure.